Skip to content

ci: move benchmark runners to c7gd.metal (Graviton3) - #9743

Closed
joseph-isaacs wants to merge 4 commits into
developfrom
claude/benchmark-runners-c7gd-metal-3mhrqb
Closed

ci: move benchmark runners to c7gd.metal (Graviton3)#9743
joseph-isaacs wants to merge 4 commits into
developfrom
claude/benchmark-runners-c7gd-metal-3mhrqb

Conversation

@joseph-isaacs

@joseph-isaacs joseph-isaacs commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Moves every benchmark workflow that ran on c6id (x86, Ice Lake) onto c7gd (arm64, Graviton3 with local NVMe). This covers the label-triggered PR benchmarks (action/bench-sql, action/bench-sql-compact, action/bench-all, compress, random access, string), the post-merge develop benchmarks, and the nightly SQL matrix. The CodSpeed and GPU benchmarks already pick their own instance families and are untouched.

Because c6id and c7gd differ in CPU architecture, the switch is not a one-word change: the build job has to move too, the runner image has to be spelled out, and the DuckDB CLI download has to become arch-aware. Results from this PR's benchmark runs will be compared against the x86 baseline on develop, so the first comparison is expected to be noisy until the develop baseline is regenerated on the new hardware.

Changes

  • pr-bench-runner.yml and develop-bench.yml: bench job runs on c7gd.metal, build job on c7gd.8xlarge so -C target-cpu=native matches the benchmark host. Both jobs pin image=ubuntu24-full-arm64-pre-v2 since bench-dedicated defaults to an x64 image.
  • sql-bench-matrix.yml: machine_type now defaults to c7gd.metal, and two new inputs, build_machine_type (default c7gd.8xlarge) and machine_image (default ubuntu24-full-arm64-pre-v2), keep the build job and image consistent with whatever family a caller picks.
  • nightly-bench.yml: matrix entry renamed from x86/c6id.metal to arm64/c7gd.metal and passes the new build/image inputs.
  • .github/actions/setup-duckdb: selects duckdb_cli-linux-{amd64,arm64}.zip from uname -m instead of hard-coding amd64.

Checks run: yamllint --strict -c .yamllint.yaml on every changed file. No Rust changes.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FUSFWCFtHuDq5uCqDgGm9i

Generated by Claude Code

@joseph-isaacs @claude
Move benchmark runners to c7gd.metal (Graviton3)
920473e 
Switch every benchmark workflow that ran on c6id off x86 and onto c7gd:
the label-triggered PR benchmarks (SQL presets, compress, random access,
string), the post-merge develop benchmarks, and the nightly SQL matrix.
The bench job runs on c7gd.metal and the binary is built on c7gd.8xlarge
so `-C target-cpu=native` matches the benchmark host. Both jobs pin the
arm64 image since `bench-dedicated` defaults to x64.

`sql-bench-matrix.yml` gains `build_machine_type` and `machine_image`
inputs alongside `machine_type` so a caller can still pick another
architecture consistently; the nightly matrix passes all three.

The setup-duckdb action now picks the DuckDB CLI archive by `uname -m`
instead of hard-coding amd64.

Signed-off-by: "Joe Isaacs" <joe.isaacs@live.co.uk>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FUSFWCFtHuDq5uCqDgGm9i
@joseph-isaacs joseph-isaacs added the action/bench-all Runs all CPU benchmarks. Do not combine with other benchmark labels; GPU is the only exception. label Sep 3, 2026 — with Claude
@github-actions github-actions Bot removed the action/bench-all Runs all CPU benchmarks. Do not combine with other benchmark labels; GPU is the only exception. label Sep 3, 2026
@codspeed-hq

codspeed-hq Bot commented Sep 3, 2026 •
edited
Loading

Copy link
Copy Markdown

Merging this PR will regress 3 benchmarks

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 2 improved benchmarks
❌ 3 regressed benchmarks
✅ 2167 untouched benchmarks
⏩ 206 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ Simulation random_i8[0.5] 67.2 µs 90.6 µs -25.87%
❌ WallTime words_gather_scalar_avx2[65536] 8.3 µs 9.4 µs -11.8%
❌ Simulation allocate_drop_bytes[0] 466 ns 520.2 ns -10.41%
⚡ WallTime arrow_checked_add_u32_neon[16384] 20.5 µs 12.7 µs +60.84%
⚡ Simulation random_i16[0.95] 94.4 µs 76.4 µs +23.64%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

Comparing claude/benchmark-runners-c7gd-metal-3mhrqb (dd8b594) with develop (792cef7)

Open in CodSpeed

Footnotes

  1. 206 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@joseph-isaacs Claude

joseph-isaacs commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

CodSpeed check: the four reported regressions are measurement noise, not this PR's. The diff touches only benchmark workflow YAML and the DuckDB CLI download in .github/actions/setup-duckdb; no Rust code changes, and codspeed.yml itself is untouched, so the CodSpeed job built and measured the same code as develop. CodSpeed's own report flags "Different runtime environments detected" on the comparison. Nothing to port or fix here; the regressions can be acknowledged in CodSpeed if the check needs to be green.

Generated by Claude Code

@github-actions

github-actions Bot commented Sep 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Polar Signals Profiling Results

Latest Run

Status Commit Job Attempt Link
🟢 Done dd8b594 fineweb-s3 2 Explore Profiling Data
🟢 Done dd8b594 tpch-s3 2 Explore Profiling Data
🟢 Done dd8b594 compress-bench 2 Explore Profiling Data
🟢 Done dd8b594 random-access-bench 1 Explore Profiling Data
🟢 Done dd8b594 statpopgen 1 Explore Profiling Data
🟢 Done dd8b594 tpch-nvme-10 1 Explore Profiling Data
🟢 Done dd8b594 clickbench-sorted-nvme 1 Explore Profiling Data
🟢 Done dd8b594 fineweb 1 Explore Profiling Data
🟢 Done dd8b594 tpch-s3-10 1 Explore Profiling Data
🟢 Done dd8b594 clickbench-nvme 1 Explore Profiling Data
🟢 Done dd8b594 tpcds-nvme 1 Explore Profiling Data
🟢 Done dd8b594 appian-nvme 1 Explore Profiling Data
🟢 Done dd8b594 tpch-nvme 1 Explore Profiling Data
🟢 Done dd8b594 polarsignals 1 Explore Profiling Data
🟢 Done dd8b594 string-bench 1 Explore Profiling Data
Previous Runs (18)
Status Commit Job Attempt Link
🟡 In Progress dd8b594 fineweb-s3 1 Explore Profiling Data
🟡 In Progress dd8b594 tpch-s3 1 Explore Profiling Data
🟡 In Progress dd8b594 compress-bench 1 Explore Profiling Data
🟢 Done 920473e random-access-bench 1 Explore Profiling Data
🟢 Done 920473e statpopgen 1 Explore Profiling Data
🟢 Done 920473e tpch-nvme-10 1 Explore Profiling Data
🟢 Done 920473e clickbench-sorted-nvme 1 Explore Profiling Data
🟢 Done 920473e tpch-s3-10 1 Explore Profiling Data
🟢 Done 920473e clickbench-nvme 1 Explore Profiling Data
🟢 Done 920473e fineweb-s3 1 Explore Profiling Data
🟢 Done 920473e fineweb 1 Explore Profiling Data
🟢 Done 920473e tpcds-nvme 1 Explore Profiling Data
🟢 Done 920473e appian-nvme 1 Explore Profiling Data
🟢 Done 920473e tpch-s3 1 Explore Profiling Data
🟢 Done 920473e tpch-nvme 1 Explore Profiling Data
🟢 Done 920473e polarsignals 1 Explore Profiling Data
🟢 Done 920473e compress-bench 1 Explore Profiling Data
🟢 Done 920473e string-bench 1 Explore Profiling Data

Powered by Polar Signals Cloud

@github-actions

github-actions Bot commented Sep 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmarks: String Encoding 📖

Commits: PR dd8b5948 vs base 792cef74

vortex / vortex-file-compressed / ms (0.820x ✅, 7↑ 0↓)
name ms (PR / base / %diff)
read/clickbench/URL/shard-0/fsst 44.339358 / 52.890143 / -16.2% 🟢
read/clickbench/URL/shard-0/onpair-12 29.958209 / 37.511877 / -20.1% 🟢
read/tpch/l_comment/fsst 78.260887 / 110.769123 / -29.3% 🟢
read/tpch/l_comment/onpair-12 80.170052 / 110.682099 / -27.6% 🟢
write/clickbench/URL/shard-0/fsst 738.360983 / 872.729637 / -15.4% 🟢
write/clickbench/URL/shard-0/onpair-12 2109.34497 / 2096.63607 / +0.6%
write/tpch/l_comment/fsst 2592.19198 / 3197.0225 / -18.9% 🟢
write/tpch/l_comment/onpair-12 3539.45452 / 4093.418 / -13.5% 🟢
vortex / vortex-file-compressed / % (1.000x ➖, 0↑ 0↓)
name % (PR / base / %diff)
size/clickbench/URL/shard-0/fsst 46.3288099 / 46.3396896 / -0.0%
size/clickbench/URL/shard-0/onpair-12 29.7417587 / 29.7417587 / +0.0%
size/tpch/l_comment/fsst 26.895123 / 26.8917906 / +0.0%
size/tpch/l_comment/onpair-12 22.6668097 / 22.6668097 / +0.0%

@joseph-isaacs Claude

joseph-isaacs commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

String Encoding benchmark failed on c7gd.metal. The runner switch itself worked: the arm64 image was picked up, the DuckDB CLI download resolved to the arm64 archive, the binary built on c7gd.8xlarge and ran on c7gd.metal. The failure is inside OnPair on the very first combination (--columns ^URL$ --encoders onpair, ClickBench hits URL column), about 6 seconds in:

Error: Other error: OnPair codes decode to more bytes than uncompressed_lengths records
  at ./benchmarks/string-bench/src/serialized.rs:273   (canonicalising the chunks read back from the serialized file)
  at ./benchmarks/string-bench/src/serialized.rs:308

onpair::try_decode_into returned OutputTooSmall, so the codes stream read back from the file decodes to more bytes than the sum of uncompressed_lengths for the same rows. The same job passes on c6id.metal on develop (latest Develop Benchmarks run 246 is green), and the arm64 CI test job (rust-test-linux-arm64) passes the vortex-onpair tests including the 100k-row roundtrip. So this is an arm64-only, data-dependent bug in code this PR does not touch, and no existing fix or issue covers it (searched for OnPair uncompressed_lengths).

What differs between the green arm64 test job and this failure, in the order I'd check them:

  1. RUSTFLAGS="-C target-cpu=native" on Graviton3 turns on SVE and LSE codegen. The CI test job builds without it.
  2. Real ClickBench URL data rather than the synthetic corpus: long strings and a dense dictionary, exercising the bit-packed codes and codes_offsets children at widths the tests may not hit.
  3. The file write/read path (vortex-file layouts and slicing of codes_offsets), which the unit tests don't go through.

I can't reproduce this from an x86 host, so I'm not pushing a fix. Suggested next step is to run the string bench on an arm64 box once without -C target-cpu=native to split (1) from (2)/(3):

cargo build -p string-bench --profile release_debug --features unstable_encodings
target/release_debug/string-bench --columns '^URL$' --encoders onpair -d gh-json -o out.json

The compress, random-access and SQL benches are still running; I'll report whether they hit the same thing.

Generated by Claude Code

@github-actions

github-actions Bot commented Sep 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmarks: Compression 📖

Commits: PR dd8b5948 vs base 792cef74

vortex / vortex-file-compressed / ns (0.898x ✅, 13↑ 2↓)
name ns (PR / base / %diff)
compress time/Arade 1082707176 / 1173457064 / -7.7%
compress time/Bimbo 4993979712 / 5062112092 / -1.3%
compress time/CMSprovider 3673374803 / 4026792967 / -8.8%
compress time/Euro2016 438171970 / 524481612 / -16.5% 🟢
compress time/Food 400126079 / 433961311 / -7.8%
compress time/HashTags 745697773 / 879075118 / -15.2% 🟢
compress time/TPC-H l_comment canonical 1007975701 / 1194031130 / -15.6% 🟢
compress time/TPC-H l_comment chunked 1019699458 / 1171588937 / -13.0% 🟢
compress time/taxi 539848659 / 557440930 / -3.2%
compress time/wide table cols=100 chunks=1 rows=1000 8367577 / 8107505 / +3.2%
compress time/wide table cols=100 chunks=50 rows=1000 8469369 / 8497328 / -0.3%
compress time/wide table cols=1000 chunks=1 rows=1000 84329459 / 84263736 / +0.1%
compress time/wide table cols=1000 chunks=50 rows=1000 85380768 / 84524961 / +1.0%
compress time/wide table cols=10000 chunks=1 rows=1000 980985656 / 980849956 / +0.0%
compress time/wide table cols=10000 chunks=50 rows=1000 984726758 / 972116618 / +1.3%
decompress time/Arade 16256662 / 26001352 / -37.5% 🟢
decompress time/Bimbo 69469557 / 91779375 / -24.3% 🟢
decompress time/CMSprovider 42199222 / 72839252 / -42.1% 🟢
decompress time/Euro2016 11322403 / 15555604 / -27.2% 🟢
decompress time/Food 4880920 / 7000742 / -30.3% 🟢
decompress time/HashTags 83668986 / 108364467 / -22.8% 🟢
decompress time/TPC-H l_comment canonical 26968788 / 34641468 / -22.1% 🟢
decompress time/TPC-H l_comment chunked 29368207 / 34924803 / -15.9% 🟢
decompress time/taxi 10567831 / 14132673 / -25.2% 🟢
decompress time/wide table cols=100 chunks=1 rows=1000 2493890 / 2342811 / +6.4%
decompress time/wide table cols=100 chunks=50 rows=1000 2507886 / 2413617 / +3.9%
decompress time/wide table cols=1000 chunks=1 rows=1000 24082018 / 22840839 / +5.4%
decompress time/wide table cols=1000 chunks=50 rows=1000 24102898 / 22587781 / +6.7%
decompress time/wide table cols=10000 chunks=1 rows=1000 298361903 / 251218578 / +18.8% 🔴
decompress time/wide table cols=10000 chunks=50 rows=1000 349338365 / 277275144 / +26.0% 🔴
vortex / vortex-file-compressed / bytes (1.000x ➖, 0↑ 0↓)
name bytes (PR / base / %diff)
vortex-file-compressed size/Arade 144460980 / 144460980 / +0.0%
vortex-file-compressed size/Bimbo 461842908 / 461842908 / +0.0%
vortex-file-compressed size/CMSprovider 415191460 / 415192460 / -0.0%
vortex-file-compressed size/Euro2016 157483620 / 157674404 / -0.1%
vortex-file-compressed size/Food 41581624 / 41581624 / +0.0%
vortex-file-compressed size/HashTags 177850636 / 177840364 / +0.0%
vortex-file-compressed size/TPC-H l_comment canonical 163461696 / 163453384 / +0.0%
vortex-file-compressed size/TPC-H l_comment chunked 163461696 / 163453384 / +0.0%
vortex-file-compressed size/taxi 51475172 / 51475172 / +0.0%
vortex-file-compressed size/wide table cols=100 chunks=1 rows=1000 932744 / 932744 / +0.0%
vortex-file-compressed size/wide table cols=100 chunks=50 rows=1000 932744 / 932744 / +0.0%
vortex-file-compressed size/wide table cols=1000 chunks=1 rows=1000 9309944 / 9309944 / +0.0%
vortex-file-compressed size/wide table cols=1000 chunks=50 rows=1000 9309944 / 9309944 / +0.0%
vortex-file-compressed size/wide table cols=10000 chunks=1 rows=1000 93117944 / 93117944 / +0.0%
vortex-file-compressed size/wide table cols=10000 chunks=50 rows=1000 93117944 / 93117944 / +0.0%
vortex / vortex-file-compressed / ratio (0.967x ➖, 9↑ 6↓)
name ratio (PR / base / %diff)
vortex:parquet-zstd ratio compress time/Arade 0.396010003 / 0.410925518 / -3.6%
vortex:parquet-zstd ratio compress time/Bimbo 0.337019187 / 0.365013892 / -7.7%
vortex:parquet-zstd ratio compress time/CMSprovider 0.467874453 / 0.524457567 / -10.8% 🟢
vortex:parquet-zstd ratio compress time/Euro2016 0.334270368 / 0.34686689 / -3.6%
vortex:parquet-zstd ratio compress time/Food 0.438716394 / 0.477408633 / -8.1%
vortex:parquet-zstd ratio compress time/HashTags 0.337734422 / 0.363981168 / -7.2%
vortex:parquet-zstd ratio compress time/TPC-H l_comment canonical 0.30740284 / 0.326458599 / -5.8%
vortex:parquet-zstd ratio compress time/TPC-H l_comment chunked 0.311631497 / 0.318213819 / -2.1%
vortex:parquet-zstd ratio compress time/taxi 0.391556247 / 0.415096165 / -5.7%
vortex:parquet-zstd ratio compress time/wide table cols=100 chunks=1 rows=1000 1.64780568 / 1.61315358 / +2.1%
vortex:parquet-zstd ratio compress time/wide table cols=100 chunks=50 rows=1000 1.68892255 / 1.68235004 / +0.4%
vortex:parquet-zstd ratio compress time/wide table cols=1000 chunks=1 rows=1000 1.57714886 / 1.35020408 / +16.8% 🔴
vortex:parquet-zstd ratio compress time/wide table cols=1000 chunks=50 rows=1000 1.58422284 / 1.34628438 / +17.7% 🔴
vortex:parquet-zstd ratio compress time/wide table cols=10000 chunks=1 rows=1000 1.74576359 / 1.32391917 / +31.9% 🔴
vortex:parquet-zstd ratio compress time/wide table cols=10000 chunks=50 rows=1000 1.7313052 / 1.30873318 / +32.3% 🔴
vortex:parquet-zstd ratio decompress time/Arade 0.0236551099 / 0.036888534 / -35.9% 🟢
vortex:parquet-zstd ratio decompress time/Bimbo 0.0348026321 / 0.0481243261 / -27.7% 🟢
vortex:parquet-zstd ratio decompress time/CMSprovider 0.0237016341 / 0.0381031485 / -37.8% 🟢
vortex:parquet-zstd ratio decompress time/Euro2016 0.0289631457 / 0.0374453672 / -22.7% 🟢
vortex:parquet-zstd ratio decompress time/Food 0.0234313616 / 0.0316199793 / -25.9% 🟢
vortex:parquet-zstd ratio decompress time/HashTags 0.138442557 / 0.158832893 / -12.8% 🟢
vortex:parquet-zstd ratio decompress time/TPC-H l_comment canonical 0.0461649111 / 0.0522666296 / -11.7% 🟢
vortex:parquet-zstd ratio decompress time/TPC-H l_comment chunked 0.0497172816 / 0.0528514094 / -5.9%
vortex:parquet-zstd ratio decompress time/taxi 0.0397882312 / 0.0498121621 / -20.1% 🟢
vortex:parquet-zstd ratio decompress time/wide table cols=100 chunks=1 rows=1000 0.96370937 / 1.01127252 / -4.7%
vortex:parquet-zstd ratio decompress time/wide table cols=100 chunks=50 rows=1000 1.01478752 / 1.0406159 / -2.5%
vortex:parquet-zstd ratio decompress time/wide table cols=1000 chunks=1 rows=1000 0.877034802 / 0.857924442 / +2.2%
vortex:parquet-zstd ratio decompress time/wide table cols=1000 chunks=50 rows=1000 0.861888282 / 0.842788026 / +2.3%
vortex:parquet-zstd ratio decompress time/wide table cols=10000 chunks=1 rows=1000 1.07073966 / 0.842556898 / +27.1% 🔴
vortex:parquet-zstd ratio decompress time/wide table cols=10000 chunks=50 rows=1000 1.23941821 / 0.917023088 / +35.2% 🔴
vortex:parquet-zstd size/Arade 0.559895285 / 0.559895285 / -0.0%
vortex:parquet-zstd size/Bimbo 1.20109789 / 1.20109789 / -0.0%
vortex:parquet-zstd size/CMSprovider 1.10163114 / 1.1016338 / -0.0%
vortex:parquet-zstd size/Euro2016 1.28057276 / 1.28212411 / -0.1%
vortex:parquet-zstd size/Food 1.16476769 / 1.16476769 / -0.0%
vortex:parquet-zstd size/HashTags 1.33223855 / 1.33216161 / +0.0%
vortex:parquet-zstd size/TPC-H l_comment canonical 1.0322273 / 1.03217481 / +0.0%
vortex:parquet-zstd size/TPC-H l_comment chunked 1.0322273 / 1.03217481 / +0.0%
vortex:parquet-zstd size/taxi 0.931110481 / 0.931110481 / -0.0%
vortex:parquet-zstd size/wide table cols=100 chunks=1 rows=1000 1.00036465 / 1.00036465 / -0.0%
vortex:parquet-zstd size/wide table cols=100 chunks=50 rows=1000 1.00036465 / 1.00036465 / -0.0%
vortex:parquet-zstd size/wide table cols=1000 chunks=1 rows=1000 0.998492064 / 0.998492064 / -0.0%
vortex:parquet-zstd size/wide table cols=1000 chunks=50 rows=1000 0.998492064 / 0.998492064 / -0.0%
vortex:parquet-zstd size/wide table cols=10000 chunks=1 rows=1000 0.998690905 / 0.998690905 / -0.0%
vortex:parquet-zstd size/wide table cols=10000 chunks=50 rows=1000 0.998690905 / 0.998690905 / -0.0%
vortex / parquet / ns (0.944x ➖, 10↑ 1↓)
name ns (PR / base / %diff)
parquet_rs-zstd compress time/Arade 2734039971 / 2855644183 / -4.3%
parquet_rs-zstd compress time/Bimbo 14818087234 / 13868272425 / +6.8%
parquet_rs-zstd compress time/CMSprovider 7851197652 / 7678014817 / +2.3%
parquet_rs-zstd compress time/Euro2016 1310831027 / 1512054415 / -13.3% 🟢
parquet_rs-zstd compress time/Food 912038129 / 908993430 / +0.3%
parquet_rs-zstd compress time/HashTags 2207941283 / 2415166486 / -8.6%
parquet_rs-zstd compress time/TPC-H l_comment canonical 3279005823 / 3657526966 / -10.3% 🟢
parquet_rs-zstd compress time/TPC-H l_comment chunked 3272132209 / 3681766365 / -11.1% 🟢
parquet_rs-zstd compress time/taxi 1378725695 / 1342919971 / +2.7%
parquet_rs-zstd compress time/wide table cols=100 chunks=1 rows=1000 5078012 / 5025873 / +1.0%
parquet_rs-zstd compress time/wide table cols=100 chunks=50 rows=1000 5014658 / 5050868 / -0.7%
parquet_rs-zstd compress time/wide table cols=1000 chunks=1 rows=1000 53469562 / 62408148 / -14.3% 🟢
parquet_rs-zstd compress time/wide table cols=1000 chunks=50 rows=1000 53894418 / 62783883 / -14.2% 🟢
parquet_rs-zstd compress time/wide table cols=10000 chunks=1 rows=1000 561923539 / 740868461 / -24.2% 🟢
parquet_rs-zstd compress time/wide table cols=10000 chunks=50 rows=1000 568777103 / 742792065 / -23.4% 🟢
parquet_rs-zstd decompress time/Arade 687236799 / 704862709 / -2.5%
parquet_rs-zstd decompress time/Bimbo 1996100665 / 1907130602 / +4.7%
parquet_rs-zstd decompress time/CMSprovider 1780435134 / 1911633418 / -6.9%
parquet_rs-zstd decompress time/Euro2016 390924491 / 415421323 / -5.9%
parquet_rs-zstd decompress time/Food 208307143 / 221402485 / -5.9%
parquet_rs-zstd decompress time/HashTags 604358860 / 682254570 / -11.4% 🟢
parquet_rs-zstd decompress time/TPC-H l_comment canonical 584183687 / 662783659 / -11.9% 🟢
parquet_rs-zstd decompress time/TPC-H l_comment chunked 590704199 / 660811195 / -10.6% 🟢
parquet_rs-zstd decompress time/taxi 265601930 / 283719325 / -6.4%
parquet_rs-zstd decompress time/wide table cols=100 chunks=1 rows=1000 2587803 / 2316696 / +11.7% 🔴
parquet_rs-zstd decompress time/wide table cols=100 chunks=50 rows=1000 2471341 / 2319412 / +6.6%
parquet_rs-zstd decompress time/wide table cols=1000 chunks=1 rows=1000 27458452 / 26623369 / +3.1%
parquet_rs-zstd decompress time/wide table cols=1000 chunks=50 rows=1000 27965223 / 26801260 / +4.3%
parquet_rs-zstd decompress time/wide table cols=10000 chunks=1 rows=1000 278650278 / 298162152 / -6.5%
parquet_rs-zstd decompress time/wide table cols=10000 chunks=50 rows=1000 281856731 / 302364409 / -6.8%
vortex / parquet / bytes (1.000x ➖, 0↑ 0↓)
name bytes (PR / base / %diff)
parquet size/Arade 258014282 / 258014282 / +0.0%
parquet size/Bimbo 384517292 / 384517292 / +0.0%
parquet size/CMSprovider 376887911 / 376887911 / +0.0%
parquet size/Euro2016 122979049 / 122979049 / +0.0%
parquet size/Food 35699500 / 35699500 / +0.0%
parquet size/HashTags 133497590 / 133497590 / +0.0%
parquet size/TPC-H l_comment canonical 158358238 / 158358238 / +0.0%
parquet size/TPC-H l_comment chunked 158358238 / 158358238 / +0.0%
parquet size/taxi 55283635 / 55283635 / +0.0%
parquet size/wide table cols=100 chunks=1 rows=1000 932404 / 932404 / +0.0%
parquet size/wide table cols=100 chunks=50 rows=1000 932404 / 932404 / +0.0%
parquet size/wide table cols=1000 chunks=1 rows=1000 9324004 / 9324004 / +0.0%
parquet size/wide table cols=1000 chunks=50 rows=1000 9324004 / 9324004 / +0.0%
parquet size/wide table cols=10000 chunks=1 rows=1000 93240004 / 93240004 / +0.0%
parquet size/wide table cols=10000 chunks=50 rows=1000 93240004 / 93240004 / +0.0%
vortex / arrow-ipc / ns (0.614x ✅, 23↑ 0↓)
name ns (PR / base / %diff)
arrow-ipc compress time/Arade 117590051 / 270802755 / -56.6% 🟢
arrow-ipc compress time/Bimbo 770405228 / 1768629459 / -56.4% 🟢
arrow-ipc compress time/CMSprovider 508458469 / 1453431634 / -65.0% 🟢
arrow-ipc compress time/Euro2016 44681811 / 98501828 / -54.6% 🟢
arrow-ipc compress time/Food 40598349 / 74079297 / -45.2% 🟢
arrow-ipc compress time/HashTags 87920935 / 291491710 / -69.8% 🟢
arrow-ipc compress time/TPC-H l_comment canonical 607215192 / 2129153939 / -71.5% 🟢
arrow-ipc compress time/TPC-H l_comment chunked 608172696 / 2121778879 / -71.3% 🟢
arrow-ipc compress time/taxi 81929619 / 160615016 / -49.0% 🟢
arrow-ipc compress time/wide table cols=100 chunks=1 rows=1000 254018 / 414441 / -38.7% 🟢
arrow-ipc compress time/wide table cols=100 chunks=50 rows=1000 248782 / 407735 / -39.0% 🟢
arrow-ipc compress time/wide table cols=1000 chunks=1 rows=1000 2640268 / 4930121 / -46.4% 🟢
arrow-ipc compress time/wide table cols=1000 chunks=50 rows=1000 2648277 / 4946489 / -46.5% 🟢
arrow-ipc compress time/wide table cols=10000 chunks=1 rows=1000 29439031 / 56556134 / -47.9% 🟢
arrow-ipc compress time/wide table cols=10000 chunks=50 rows=1000 29197311 / 56649310 / -48.5% 🟢
arrow-ipc decompress time/Arade 196177692 / 185753923 / +5.6%
arrow-ipc decompress time/Bimbo 467022929 / 592638199 / -21.2% 🟢
arrow-ipc decompress time/CMSprovider 662500168 / 715741545 / -7.4%
arrow-ipc decompress time/Euro2016 85069963 / 96372573 / -11.7% 🟢
arrow-ipc decompress time/Food 62461023 / 65549547 / -4.7%
arrow-ipc decompress time/HashTags 169741835 / 209955386 / -19.2% 🟢
arrow-ipc decompress time/TPC-H l_comment canonical 633196730 / 1033293616 / -38.7% 🟢
arrow-ipc decompress time/TPC-H l_comment chunked 631304656 / 1046224757 / -39.7% 🟢
arrow-ipc decompress time/taxi 52422501 / 72221291 / -27.4% 🟢
arrow-ipc decompress time/wide table cols=100 chunks=1 rows=1000 386068 / 390724 / -1.2%
arrow-ipc decompress time/wide table cols=100 chunks=50 rows=1000 382008 / 395360 / -3.4%
arrow-ipc decompress time/wide table cols=1000 chunks=1 rows=1000 3903795 / 4140209 / -5.7%
arrow-ipc decompress time/wide table cols=1000 chunks=50 rows=1000 3926319 / 4065180 / -3.4%
arrow-ipc decompress time/wide table cols=10000 chunks=1 rows=1000 40120794 / 50926584 / -21.2% 🟢
arrow-ipc decompress time/wide table cols=10000 chunks=50 rows=1000 39755481 / 50569984 / -21.4% 🟢
vortex / arrow-ipc / bytes (1.000x ➖, 0↑ 0↓)
name bytes (PR / base / %diff)
arrow-ipc size/Arade 691765162 / 691765162 / +0.0%
arrow-ipc size/Bimbo 3279590114 / 3279590114 / +0.0%
arrow-ipc size/CMSprovider 2422259682 / 2422259682 / +0.0%
arrow-ipc size/Euro2016 360582666 / 360582666 / +0.0%
arrow-ipc size/Food 255782338 / 255782338 / +0.0%
arrow-ipc size/HashTags 704785810 / 704785810 / +0.0%
arrow-ipc size/TPC-H l_comment canonical 4852929378 / 4852929378 / +0.0%
arrow-ipc size/TPC-H l_comment chunked 4852929378 / 4852929378 / +0.0%
arrow-ipc size/taxi 482329690 / 482329690 / +0.0%
arrow-ipc size/wide table cols=100 chunks=1 rows=1000 1257290 / 1257290 / +0.0%
arrow-ipc size/wide table cols=100 chunks=50 rows=1000 1257290 / 1257290 / +0.0%
arrow-ipc size/wide table cols=1000 chunks=1 rows=1000 12568506 / 12568506 / +0.0%
arrow-ipc size/wide table cols=1000 chunks=50 rows=1000 12568506 / 12568506 / +0.0%
arrow-ipc size/wide table cols=10000 chunks=1 rows=1000 125752506 / 125752506 / +0.0%
arrow-ipc size/wide table cols=10000 chunks=50 rows=1000 125752506 / 125752506 / +0.0%

@github-actions

github-actions Bot commented Sep 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmarks: PolarSignals Profiling 📖

Commits: PR dd8b5948 vs base 792cef74
Vortex (geomean): hot 0.636x ✅ · cold 0.637x ✅

datafusion / vortex-file-compressed / ns (0.636x ✅, 9↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
polarsignals_q00/datafusion:vortex-file-compressed 50329490 / 105205168 / -52.2% 🟢 105870564 / 206893739 / -48.8% 🟢 0.48 / 0.51 / -6.5%
polarsignals_q01/datafusion:vortex-file-compressed 77332311 / 156173542 / -50.5% 🟢 158144779 / 270983613 / -41.6% 🟢 0.49 / 0.58 / -15.2%
polarsignals_q02/datafusion:vortex-file-compressed 39217147 / 50728419 / -22.7% 🟢 88023209 / 122729518 / -28.3% 🟢 0.45 / 0.41 / +7.8%
polarsignals_q03/datafusion:vortex-file-compressed 63882514 / 165198648 / -61.3% 🟢 160097760 / 277824786 / -42.4% 🟢 0.40 / 0.59 / -32.9%
polarsignals_q04/datafusion:vortex-file-compressed 9061910 / 10417590 / -13.0% 🟢 54408210 / 78387071 / -30.6% 🟢 0.17 / 0.13 / +25.3%
polarsignals_q05/datafusion:vortex-file-compressed 11526278 / 13993408 / -17.6% 🟢 64250647 / 91748910 / -30.0% 🟢 0.18 / 0.15 / +17.6%
polarsignals_q06/datafusion:vortex-file-compressed 22203047 / 30355786 / -26.9% 🟢 73735447 / 102083230 / -27.8% 🟢 0.30 / 0.30 / +1.3%
polarsignals_q07/datafusion:vortex-file-compressed 11045194 / 12076840 / -8.5% 61385561 / 88348875 / -30.5% 🟢 0.18 / 0.14 / +31.6%
polarsignals_q08/datafusion:vortex-file-compressed 116782190 / 287836716 / -59.4% 🟢 217002067 / 403337633 / -46.2% 🟢 0.54 / 0.71 / -24.6%
polarsignals_q09/datafusion:vortex-file-compressed 9315416 / 11810643 / -21.1% 🟢 57989170 / 85284053 / -32.0% 🟢 0.16 / 0.14 / +16.0%

No file size changes detected.

@joseph-isaacs Claude

joseph-isaacs commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Compression benchmark failed on c7gd.metal too, in a different codec. taxi passed all three formats; Arade failed reading back the vortex-file-compressed output:

Error: decompressing Arade as vortex-file-compressed
Caused by:
    0: External error: FSST decoded 441 bytes, expected 208
       at ./encodings/fsst/src/canonical.rs:107

That is the same shape of failure as the OnPair one above: the decoded byte count disagrees with the uncompressed_lengths child read back from a Vortex file, now in FSST rather than OnPair. Two independent string codecs, neither with any arm64-specific code, failing identically on their first real dataset makes a per-codec bug unlikely. The shared pieces are what both decode plans consume from the file: the bit-packed uncompressed_lengths and codes/offsets children (FastLanes unpack, which is generic auto-vectorised code), and the file read path. Both bench binaries are built with RUSTFLAGS="-C target-cpu=native", which on Graviton3 means SVE codegen, whereas the green rust-test-linux-arm64 job builds without it.

So the working theory is a miscompile or exposed UB under -C target-cpu=native on Graviton3 in the FastLanes unpack path, rather than anything in OnPair or FSST. The cheapest discriminator is still the one above: build the string or compress bench on an arm64 box without -C target-cpu=native and re-run. If it passes, the follow-up is a -C target-cpu=neoverse-v1 vs -C target-feature=-sve bisection. I'm not pushing a fix from this PR since it would be a change to benchmark methodology (dropping target-cpu=native) or to code the PR doesn't touch, and I can't verify either from an x86 host.

Generated by Claude Code

@github-actions

github-actions Bot commented Sep 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-H SF=1 on NVME 📖

Commits: PR dd8b5948 vs base 792cef74
Verdict: Likely improvement (high confidence)
Attributed Vortex impact: -22.9%
Engines: DataFusion No clear signal (-11.2%, environment too noisy confidence) · DuckDB Likely improvement (-33.0%, high confidence)
Vortex (geomean): hot 0.690x ✅ · cold 0.600x ✅
Parquet (geomean): hot 0.895x ✅ · cold 0.829x ✅
Shifts: Parquet (control) -10.5% · Median polish -27.6%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Table cells: Each cell reads PR / base / %diff. The hot and cold columns mark a change as 🔴 slower or 🟢 faster once it clears this suite's threshold; anything smaller is noise here and is left unmarked, as is hot/cold, because a shift in the warm-up ratio is not a win or a loss by itself.
datafusion / vortex-file-compressed / ns (0.728x ✅, 22↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/datafusion:vortex-file-compressed 21964538 / 42868469 / -48.8% 🟢 65330309 / 97104348 / -32.7% 🟢 0.34 / 0.44 / -23.8%
tpch_q02/datafusion:vortex-file-compressed 17910095 / 23002305 / -22.1% 🟢 46464905 / 66741007 / -30.4% 🟢 0.39 / 0.34 / +11.8%
tpch_q03/datafusion:vortex-file-compressed 19719761 / 27444196 / -28.1% 🟢 60704770 / 97798658 / -37.9% 🟢 0.32 / 0.28 / +15.8%
tpch_q04/datafusion:vortex-file-compressed 11704716 / 15455840 / -24.3% 🟢 44428648 / 64614083 / -31.2% 🟢 0.26 / 0.24 / +10.1%
tpch_q05/datafusion:vortex-file-compressed 34503392 / 47300825 / -27.1% 🟢 79167713 / 117890748 / -32.8% 🟢 0.44 / 0.40 / +8.6%
tpch_q06/datafusion:vortex-file-compressed 6483925 / 8296410 / -21.8% 🟢 42197206 / 52474676 / -19.6% 🟢 0.15 / 0.16 / -2.8%
tpch_q07/datafusion:vortex-file-compressed 40227045 / 55849622 / -28.0% 🟢 86497946 / 127818709 / -32.3% 🟢 0.47 / 0.44 / +6.4%
tpch_q08/datafusion:vortex-file-compressed 36467249 / 45790245 / -20.4% 🟢 78413268 / 116708810 / -32.8% 🟢 0.47 / 0.39 / +18.5%
tpch_q09/datafusion:vortex-file-compressed 38588946 / 54253742 / -28.9% 🟢 81554083 / 132675967 / -38.5% 🟢 0.47 / 0.41 / +15.7%
tpch_q10/datafusion:vortex-file-compressed 19564884 / 26691101 / -26.7% 🟢 63240698 / 99334336 / -36.3% 🟢 0.31 / 0.27 / +15.1%
tpch_q11/datafusion:vortex-file-compressed 13759321 / 17764445 / -22.5% 🟢 44273482 / 60856494 / -27.2% 🟢 0.31 / 0.29 / +6.5%
tpch_q12/datafusion:vortex-file-compressed 19307565 / 24335110 / -20.7% 🟢 57795416 / 74459139 / -22.4% 🟢 0.33 / 0.33 / +2.2%
tpch_q13/datafusion:vortex-file-compressed 17437149 / 24011084 / -27.4% 🟢 46641286 / 73860252 / -36.9% 🟢 0.37 / 0.33 / +15.0%
tpch_q14/datafusion:vortex-file-compressed 12151823 / 16618472 / -26.9% 🟢 51936202 / 69186907 / -24.9% 🟢 0.23 / 0.24 / -2.6%
tpch_q15/datafusion:vortex-file-compressed 15434526 / 20789244 / -25.8% 🟢 58088155 / 80253545 / -27.6% 🟢 0.27 / 0.26 / +2.6%
tpch_q16/datafusion:vortex-file-compressed 16439096 / 22117863 / -25.7% 🟢 44526684 / 74964492 / -40.6% 🟢 0.37 / 0.30 / +25.1%
tpch_q17/datafusion:vortex-file-compressed 37533894 / 55906461 / -32.9% 🟢 84157417 / 121659043 / -30.8% 🟢 0.45 / 0.46 / -2.9%
tpch_q18/datafusion:vortex-file-compressed 40235865 / 62710053 / -35.8% 🟢 73410941 / 122114328 / -39.9% 🟢 0.55 / 0.51 / +6.7%
tpch_q19/datafusion:vortex-file-compressed 14518555 / 16429937 / -11.6% 🟢 55378458 / 73104365 / -24.2% 🟢 0.26 / 0.22 / +16.7%
tpch_q20/datafusion:vortex-file-compressed 19509903 / 27268591 / -28.5% 🟢 56784522 / 87840618 / -35.4% 🟢 0.34 / 0.31 / +10.7%
tpch_q21/datafusion:vortex-file-compressed 40547366 / 60409732 / -32.9% 🟢 81692153 / 132203653 / -38.2% 🟢 0.50 / 0.46 / +8.6%
tpch_q22/datafusion:vortex-file-compressed 10154181 / 13206072 / -23.1% 🟢 43304228 / 62332374 / -30.5% 🟢 0.23 / 0.21 / +10.7%
datafusion / vortex-compact / ns (0.699x ✅, 22↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/datafusion:vortex-compact 25803496 / 47067564 / -45.2% 🟢 67708675 / 101871735 / -33.5% 🟢 0.38 / 0.46 / -17.5%
tpch_q02/datafusion:vortex-compact 20039146 / 27166192 / -26.2% 🟢 48330002 / 70609200 / -31.6% 🟢 0.41 / 0.38 / +7.8%
tpch_q03/datafusion:vortex-compact 20729565 / 28821313 / -28.1% 🟢 60010271 / 91091510 / -34.1% 🟢 0.35 / 0.32 / +9.2%
tpch_q04/datafusion:vortex-compact 13035511 / 18433752 / -29.3% 🟢 44694698 / 68158547 / -34.4% 🟢 0.29 / 0.27 / +7.8%
tpch_q05/datafusion:vortex-compact 34951496 / 48703369 / -28.2% 🟢 75307677 / 124449225 / -39.5% 🟢 0.46 / 0.39 / +18.6%
tpch_q06/datafusion:vortex-compact 7836015 / 10048465 / -22.0% 🟢 40916777 / 58183156 / -29.7% 🟢 0.19 / 0.17 / +10.9%
tpch_q07/datafusion:vortex-compact 40357742 / 58317331 / -30.8% 🟢 83921647 / 127076211 / -34.0% 🟢 0.48 / 0.46 / +4.8%
tpch_q08/datafusion:vortex-compact 39345499 / 49558009 / -20.6% 🟢 80045925 / 119415250 / -33.0% 🟢 0.49 / 0.42 / +18.4%
tpch_q09/datafusion:vortex-compact 39858288 / 57351580 / -30.5% 🟢 85689097 / 130277121 / -34.2% 🟢 0.47 / 0.44 / +5.7%
tpch_q10/datafusion:vortex-compact 22430172 / 33841810 / -33.7% 🟢 65495561 / 95179562 / -31.2% 🟢 0.34 / 0.36 / -3.7%
tpch_q11/datafusion:vortex-compact 15310513 / 21135937 / -27.6% 🟢 44229092 / 66648861 / -33.6% 🟢 0.35 / 0.32 / +9.2%
tpch_q12/datafusion:vortex-compact 22000205 / 29967156 / -26.6% 🟢 53539446 / 79745999 / -32.9% 🟢 0.41 / 0.38 / +9.3%
tpch_q13/datafusion:vortex-compact 18675240 / 27034559 / -30.9% 🟢 47670261 / 76595406 / -37.8% 🟢 0.39 / 0.35 / +11.0%
tpch_q14/datafusion:vortex-compact 13562116 / 18377266 / -26.2% 🟢 51402262 / 74594094 / -31.1% 🟢 0.26 / 0.25 / +7.1%
tpch_q15/datafusion:vortex-compact 19069032 / 25772694 / -26.0% 🟢 56685306 / 81944307 / -30.8% 🟢 0.34 / 0.31 / +7.0%
tpch_q16/datafusion:vortex-compact 17763576 / 24968820 / -28.9% 🟢 46840867 / 73919964 / -36.6% 🟢 0.38 / 0.34 / +12.3%
tpch_q17/datafusion:vortex-compact 38838708 / 56908368 / -31.8% 🟢 83154460 / 123014006 / -32.4% 🟢 0.47 / 0.46 / +1.0%
tpch_q18/datafusion:vortex-compact 40206836 / 64413325 / -37.6% 🟢 73551087 / 121000861 / -39.2% 🟢 0.55 / 0.53 / +2.7%
tpch_q19/datafusion:vortex-compact 21996576 / 32838892 / -33.0% 🟢 58348354 / 90361674 / -35.4% 🟢 0.38 / 0.36 / +3.7%
tpch_q20/datafusion:vortex-compact 21064881 / 30296821 / -30.5% 🟢 61236072 / 85407495 / -28.3% 🟢 0.34 / 0.35 / -3.0%
tpch_q21/datafusion:vortex-compact 43120812 / 64896362 / -33.6% 🟢 83389510 / 145167933 / -42.6% 🟢 0.52 / 0.45 / +15.7%
tpch_q22/datafusion:vortex-compact 10225448 / 14817509 / -31.0% 🟢 41818557 / 61560459 / -32.1% 🟢 0.24 / 0.24 / +1.6%
datafusion / parquet / ns (0.804x ✅, 22↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/datafusion:parquet 78166286 / 109894603 / -28.9% 🟢 108637629 / 197978050 / -45.1% 🟢 0.72 / 0.56 / +29.6%
tpch_q02/datafusion:parquet 58386777 / 69168999 / -15.6% 🟢 87819363 / 122507974 / -28.3% 🟢 0.66 / 0.56 / +17.8%
tpch_q03/datafusion:parquet 74147013 / 89944626 / -17.6% 🟢 114672115 / 144908358 / -20.9% 🟢 0.65 / 0.62 / +4.2%
tpch_q04/datafusion:parquet 43310852 / 50414323 / -14.1% 🟢 78409969 / 104713438 / -25.1% 🟢 0.55 / 0.48 / +14.7%
tpch_q05/datafusion:parquet 84847106 / 111605701 / -24.0% 🟢 129721074 / 169559163 / -23.5% 🟢 0.65 / 0.66 / -0.6%
tpch_q06/datafusion:parquet 32436635 / 46835259 / -30.7% 🟢 64147522 / 101622566 / -36.9% 🟢 0.51 / 0.46 / +9.7%
tpch_q07/datafusion:parquet 90518807 / 114851419 / -21.2% 🟢 136051954 / 199937656 / -32.0% 🟢 0.67 / 0.57 / +15.8%
tpch_q08/datafusion:parquet 93151712 / 106452089 / -12.5% 🟢 141235939 / 180786070 / -21.9% 🟢 0.66 / 0.59 / +12.0%
tpch_q09/datafusion:parquet 114590303 / 134322345 / -14.7% 🟢 166933946 / 222256726 / -24.9% 🟢 0.69 / 0.60 / +13.6%
tpch_q10/datafusion:parquet 94580119 / 113552850 / -16.7% 🟢 139945728 / 199482800 / -29.8% 🟢 0.68 / 0.57 / +18.7%
tpch_q11/datafusion:parquet 42736095 / 51603712 / -17.2% 🟢 74389871 / 107033463 / -30.5% 🟢 0.57 / 0.48 / +19.2%
tpch_q12/datafusion:parquet 69978956 / 98928964 / -29.3% 🟢 108093268 / 153632061 / -29.6% 🟢 0.65 / 0.64 / +0.5%
tpch_q13/datafusion:parquet 185208530 / 213197708 / -13.1% 🟢 218124138 / 267399118 / -18.4% 🟢 0.85 / 0.80 / +6.5%
tpch_q14/datafusion:parquet 39267433 / 48369627 / -18.8% 🟢 80004395 / 108948466 / -26.6% 🟢 0.49 / 0.44 / +10.6%
tpch_q15/datafusion:parquet 51739778 / 67299367 / -23.1% 🟢 88064757 / 123235714 / -28.5% 🟢 0.59 / 0.55 / +7.6%
tpch_q16/datafusion:parquet 41500417 / 51044475 / -18.7% 🟢 69384040 / 103438336 / -32.9% 🟢 0.60 / 0.49 / +21.2%
tpch_q17/datafusion:parquet 133295197 / 149830598 / -11.0% 🟢 176545762 / 211734878 / -16.6% 🟢 0.76 / 0.71 / +6.7%
tpch_q18/datafusion:parquet 123288515 / 161357248 / -23.6% 🟢 161000057 / 229948917 / -30.0% 🟢 0.77 / 0.70 / +9.1%
tpch_q19/datafusion:parquet 61308578 / 72892305 / -15.9% 🟢 105343786 / 127957644 / -17.7% 🟢 0.58 / 0.57 / +2.2%
tpch_q20/datafusion:parquet 59717417 / 78352365 / -23.8% 🟢 103254352 / 136239786 / -24.2% 🟢 0.58 / 0.58 / +0.6%
tpch_q21/datafusion:parquet 102075777 / 128035207 / -20.3% 🟢 144980714 / 205037767 / -29.3% 🟢 0.70 / 0.62 / +12.8%
tpch_q22/datafusion:parquet 45999022 / 55139914 / -16.6% 🟢 77267938 / 103901043 / -25.6% 🟢 0.60 / 0.53 / +12.2%
duckdb / vortex-file-compressed / ns (0.658x ✅, 22↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/duckdb:vortex-file-compressed 13441925 / 23886767 / -43.7% 🟢 31331052 / 64611979 / -51.5% 🟢 0.43 / 0.37 / +16.0%
tpch_q02/duckdb:vortex-file-compressed 20380034 / 29566545 / -31.1% 🟢 33317506 / 48488399 / -31.3% 🟢 0.61 / 0.61 / +0.3%
tpch_q03/duckdb:vortex-file-compressed 22681164 / 34572127 / -34.4% 🟢 43078916 / 88932249 / -51.6% 🟢 0.53 / 0.39 / +35.4%
tpch_q04/duckdb:vortex-file-compressed 20158741 / 30600052 / -34.1% 🟢 36014336 / 73393944 / -50.9% 🟢 0.56 / 0.42 / +34.3%
tpch_q05/duckdb:vortex-file-compressed 26948524 / 40275938 / -33.1% 🟢 44908726 / 78814484 / -43.0% 🟢 0.60 / 0.51 / +17.4%
tpch_q06/duckdb:vortex-file-compressed 6382151 / 9593391 / -33.5% 🟢 21090456 / 42443325 / -50.3% 🟢 0.30 / 0.23 / +33.9%
tpch_q07/duckdb:vortex-file-compressed 24793825 / 40304505 / -38.5% 🟢 44201325 / 82292817 / -46.3% 🟢 0.56 / 0.49 / +14.5%
tpch_q08/duckdb:vortex-file-compressed 28022337 / 45493026 / -38.4% 🟢 51763122 / 90433081 / -42.8% 🟢 0.54 / 0.50 / +7.6%
tpch_q09/duckdb:vortex-file-compressed 37965387 / 57720124 / -34.2% 🟢 63734797 / 105426171 / -39.5% 🟢 0.60 / 0.55 / +8.8%
tpch_q10/duckdb:vortex-file-compressed 32812593 / 47675945 / -31.2% 🟢 59543215 / 95917890 / -37.9% 🟢 0.55 / 0.50 / +10.9%
tpch_q11/duckdb:vortex-file-compressed 9572261 / 18098749 / -47.1% 🟢 19163105 / 37411144 / -48.8% 🟢 0.50 / 0.48 / +3.3%
tpch_q12/duckdb:vortex-file-compressed 17895074 / 26091159 / -31.4% 🟢 33926366 / 60850466 / -44.2% 🟢 0.53 / 0.43 / +23.0%
tpch_q13/duckdb:vortex-file-compressed 27997813 / 42528081 / -34.2% 🟢 44291961 / 123321064 / -64.1% 🟢 0.63 / 0.34 / +83.3%
tpch_q14/duckdb:vortex-file-compressed 16164561 / 21821321 / -25.9% 🟢 35501883 / 56677043 / -37.4% 🟢 0.46 / 0.39 / +18.3%
tpch_q15/duckdb:vortex-file-compressed 11330002 / 17743672 / -36.1% 🟢 25847608 / 54745584 / -52.8% 🟢 0.44 / 0.32 / +35.2%
tpch_q16/duckdb:vortex-file-compressed 23143368 / 30870593 / -25.0% 🟢 31290664 / 46605097 / -32.9% 🟢 0.74 / 0.66 / +11.7%
tpch_q17/duckdb:vortex-file-compressed 18544899 / 29895372 / -38.0% 🟢 38062246 / 74655882 / -49.0% 🟢 0.49 / 0.40 / +21.7%
tpch_q18/duckdb:vortex-file-compressed 32103841 / 48892175 / -34.3% 🟢 47456097 / 79866774 / -40.6% 🟢 0.68 / 0.61 / +10.5%
tpch_q19/duckdb:vortex-file-compressed 26292262 / 35901675 / -26.8% 🟢 47735306 / 78682732 / -39.3% 🟢 0.55 / 0.46 / +20.7%
tpch_q20/duckdb:vortex-file-compressed 21750062 / 33725324 / -35.5% 🟢 40531454 / 82027545 / -50.6% 🟢 0.54 / 0.41 / +30.5%
tpch_q21/duckdb:vortex-file-compressed 82454722 / 119130752 / -30.8% 🟢 98923058 / 158801724 / -37.7% 🟢 0.83 / 0.75 / +11.1%
tpch_q22/duckdb:vortex-file-compressed 14753192 / 21439806 / -31.2% 🟢 26307171 / 41985166 / -37.3% 🟢 0.56 / 0.51 / +9.8%
duckdb / vortex-compact / ns (0.678x ✅, 22↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/duckdb:vortex-compact 19571226 / 27572406 / -29.0% 🟢 33846078 / 68382463 / -50.5% 🟢 0.58 / 0.40 / +43.4%
tpch_q02/duckdb:vortex-compact 23284569 / 34267109 / -32.0% 🟢 32775138 / 54868137 / -40.3% 🟢 0.71 / 0.62 / +13.8%
tpch_q03/duckdb:vortex-compact 23930930 / 38290081 / -37.5% 🟢 41160806 / 81797249 / -49.7% 🟢 0.58 / 0.47 / +24.2%
tpch_q04/duckdb:vortex-compact 23515369 / 33546863 / -29.9% 🟢 37972827 / 68182906 / -44.3% 🟢 0.62 / 0.49 / +25.9%
tpch_q05/duckdb:vortex-compact 29224131 / 43845607 / -33.3% 🟢 47618418 / 82697041 / -42.4% 🟢 0.61 / 0.53 / +15.8%
tpch_q06/duckdb:vortex-compact 8029403 / 10199157 / -21.3% 🟢 20743134 / 43698879 / -52.5% 🟢 0.39 / 0.23 / +65.8%
tpch_q07/duckdb:vortex-compact 27550882 / 39685164 / -30.6% 🟢 50221054 / 83881202 / -40.1% 🟢 0.55 / 0.47 / +16.0%
tpch_q08/duckdb:vortex-compact 30946820 / 49406161 / -37.4% 🟢 52366443 / 97244745 / -46.1% 🟢 0.59 / 0.51 / +16.3%
tpch_q09/duckdb:vortex-compact 40846415 / 60460170 / -32.4% 🟢 64372409 / 109635382 / -41.3% 🟢 0.63 / 0.55 / +15.1%
tpch_q10/duckdb:vortex-compact 34981728 / 52024632 / -32.8% 🟢 58491658 / 95702653 / -38.9% 🟢 0.60 / 0.54 / +10.0%
tpch_q11/duckdb:vortex-compact 11035387 / 19011577 / -42.0% 🟢 20008461 / 40704844 / -50.8% 🟢 0.55 / 0.47 / +18.1%
tpch_q12/duckdb:vortex-compact 22068399 / 29214238 / -24.5% 🟢 33639454 / 72110330 / -53.4% 🟢 0.66 / 0.41 / +61.9%
tpch_q13/duckdb:vortex-compact 29689900 / 42337192 / -29.9% 🟢 45571455 / 139563652 / -67.3% 🟢 0.65 / 0.30 / +114.8%
tpch_q14/duckdb:vortex-compact 18442957 / 24850711 / -25.8% 🟢 36576033 / 67382989 / -45.7% 🟢 0.50 / 0.37 / +36.7%
tpch_q15/duckdb:vortex-compact 12526242 / 18098003 / -30.8% 🟢 31816982 / 76536464 / -58.4% 🟢 0.39 / 0.24 / +66.5%
tpch_q16/duckdb:vortex-compact 24795998 / 32115608 / -22.8% 🟢 32124029 / 48402493 / -33.6% 🟢 0.77 / 0.66 / +16.3%
tpch_q17/duckdb:vortex-compact 20620154 / 35830587 / -42.5% 🟢 43280538 / 92731281 / -53.3% 🟢 0.48 / 0.39 / +23.3%
tpch_q18/duckdb:vortex-compact 34041157 / 57266225 / -40.6% 🟢 45617695 / 94800256 / -51.9% 🟢 0.75 / 0.60 / +23.5%
tpch_q19/duckdb:vortex-compact 28531877 / 42045032 / -32.1% 🟢 48699108 / 76858431 / -36.6% 🟢 0.59 / 0.55 / +7.1%
tpch_q20/duckdb:vortex-compact 24967666 / 37977846 / -34.3% 🟢 48388209 / 92845156 / -47.9% 🟢 0.52 / 0.41 / +26.1%
tpch_q21/duckdb:vortex-compact 86688716 / 123864245 / -30.0% 🟢 99520766 / 165854429 / -40.0% 🟢 0.87 / 0.75 / +16.6%
tpch_q22/duckdb:vortex-compact 15397943 / 22731097 / -32.3% 🟢 26713021 / 41815861 / -36.1% 🟢 0.58 / 0.54 / +6.0%
duckdb / parquet / ns (0.996x ➖, 1↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/duckdb:parquet 72069404 / 81972126 / -12.1% 🟢 81148918 / 92775689 / -12.5% 🟢 0.89 / 0.88 / +0.5%
tpch_q02/duckdb:parquet 44594649 / 46131844 / -3.3% 56532060 / 62906236 / -10.1% 🟢 0.79 / 0.73 / +7.6%
tpch_q03/duckdb:parquet 82007388 / 80997314 / +1.2% 99434609 / 100834209 / -1.4% 0.82 / 0.80 / +2.7%
tpch_q04/duckdb:parquet 57649970 / 58701813 / -1.8% 71858250 / 74575501 / -3.6% 0.80 / 0.79 / +1.9%
tpch_q05/duckdb:parquet 81121885 / 79166658 / +2.5% 97324983 / 119048243 / -18.2% 🟢 0.83 / 0.66 / +25.3%
tpch_q06/duckdb:parquet 25422293 / 25047904 / +1.5% 32041034 / 33789164 / -5.2% 0.79 / 0.74 / +7.0%
tpch_q07/duckdb:parquet 81512483 / 80449788 / +1.3% 99983966 / 98645161 / +1.4% 0.82 / 0.82 / -0.0%
tpch_q08/duckdb:parquet 97013962 / 97397123 / -0.4% 115977976 / 123349833 / -6.0% 0.84 / 0.79 / +5.9%
tpch_q09/duckdb:parquet 150855625 / 147329994 / +2.4% 170365277 / 171199977 / -0.5% 0.89 / 0.86 / +2.9%
tpch_q10/duckdb:parquet 138528786 / 141735604 / -2.3% 160852433 / 163965460 / -1.9% 0.86 / 0.86 / -0.4%
tpch_q11/duckdb:parquet 27533138 / 25310553 / +8.8% 33973999 / 33710870 / +0.8% 0.81 / 0.75 / +7.9%
tpch_q12/duckdb:parquet 53764582 / 53378536 / +0.7% 65144933 / 67990881 / -4.2% 0.83 / 0.79 / +5.1%
tpch_q13/duckdb:parquet 309158591 / 286320706 / +8.0% 317187749 / 304372126 / +4.2% 0.97 / 0.94 / +3.6%
tpch_q14/duckdb:parquet 55818698 / 56173359 / -0.6% 68898711 / 72483974 / -4.9% 0.81 / 0.77 / +4.5%
tpch_q15/duckdb:parquet 31893025 / 30715503 / +3.8% 41858634 / 42790848 / -2.2% 0.76 / 0.72 / +6.1%
tpch_q16/duckdb:parquet 67380341 / 66617167 / +1.1% 77180946 / 83511429 / -7.6% 0.87 / 0.80 / +9.4%
tpch_q17/duckdb:parquet 57453739 / 58070245 / -1.1% 70749562 / 76698551 / -7.8% 0.81 / 0.76 / +7.3%
tpch_q18/duckdb:parquet 116522187 / 128435482 / -9.3% 129700100 / 140221185 / -7.5% 0.90 / 0.92 / -1.9%
tpch_q19/duckdb:parquet 78496129 / 78063652 / +0.6% 91689219 / 96267279 / -4.8% 0.86 / 0.81 / +5.6%
tpch_q20/duckdb:parquet 75851400 / 77807904 / -2.5% 91704943 / 100979552 / -9.2% 0.83 / 0.77 / +7.3%
tpch_q21/duckdb:parquet 180241749 / 189638793 / -5.0% 194698655 / 203600243 / -4.4% 0.93 / 0.93 / -0.6%
tpch_q22/duckdb:parquet 64131860 / 64110568 / +0.0% 72926759 / 79878591 / -8.7% 0.88 / 0.80 / +9.6%
File Size Changes (5 files changed, +0.0% overall, 4↑ 1↓)
File Scale Format Base HEAD Change %
part.vortex 1.0 vortex-file-compressed 4.85 MB 4.87 MB +23.20 KB +0.5%
orders.vortex 1.0 vortex-file-compressed 35.92 MB 35.95 MB +35.85 KB +0.1%
lineitem.vortex 1.0 vortex-file-compressed 155.88 MB 155.89 MB +8.12 KB +0.0%
partsupp.vortex 1.0 vortex-file-compressed 23.60 MB 23.60 MB +16 B +0.0%
customer.vortex 1.0 vortex-file-compressed 8.35 MB 8.35 MB 48 B -0.0%

Totals:

  • vortex-compact: 189.71 MB → 189.71 MB (0.0%)
  • vortex-file-compressed: 229.43 MB → 229.50 MB (+0.0%)

@github-actions

github-actions Bot commented Sep 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmarks: FineWeb NVMe 📖

Commits: PR dd8b5948 vs base 792cef74
Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: -23.9%
Engines: DataFusion No clear signal (-10.0%, environment too noisy confidence) · DuckDB Likely improvement (-35.6%, environment too noisy confidence)
Vortex (geomean): hot 0.543x ✅ · cold 0.799x ✅
Parquet (geomean): hot 0.714x ✅ · cold 0.986x ➖
Shifts: Parquet (control) -28.6% · Median polish -42.1%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Table cells: Each cell reads PR / base / %diff. The hot and cold columns mark a change as 🔴 slower or 🟢 faster once it clears this suite's threshold; anything smaller is noise here and is left unmarked, as is hot/cold, because a shift in the warm-up ratio is not a win or a loss by itself.
datafusion / vortex-file-compressed / ns (0.592x ✅, 9↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
fineweb_q00/datafusion:vortex-file-compressed 5580985 / 8817729 / -36.7% 🟢 25777603 / 36863716 / -30.1% 🟢 0.22 / 0.24 / -9.5%
fineweb_q01/datafusion:vortex-file-compressed 15904037 / 27916070 / -43.0% 🟢 89397769 / 94799603 / -5.7% 0.18 / 0.29 / -39.6%
fineweb_q02/datafusion:vortex-file-compressed 15205912 / 27056670 / -43.8% 🟢 97264231 / 91044497 / +6.8% 0.16 / 0.30 / -47.4%
fineweb_q03/datafusion:vortex-file-compressed 27023610 / 44951024 / -39.9% 🟢 245649865 / 172456620 / +42.4% 🔴 0.11 / 0.26 / -57.8%
fineweb_q04/datafusion:vortex-file-compressed 116820789 / 199126303 / -41.3% 🟢 301041532 / 304065350 / -1.0% 0.39 / 0.65 / -40.7%
fineweb_q05/datafusion:vortex-file-compressed 87827114 / 154663474 / -43.2% 🟢 269651135 / 280201436 / -3.8% 0.33 / 0.55 / -41.0%
fineweb_q06/datafusion:vortex-file-compressed 24169623 / 40287400 / -40.0% 🟢 246068291 / 190356458 / +29.3% 🔴 0.10 / 0.21 / -53.6%
fineweb_q07/datafusion:vortex-file-compressed 25039004 / 38776777 / -35.4% 🟢 239591019 / 182797109 / +31.1% 🔴 0.10 / 0.21 / -50.7%
fineweb_q08/datafusion:vortex-file-compressed 11711162 / 20535396 / -43.0% 🟢 103297796 / 82329456 / +25.5% 🔴 0.11 / 0.25 / -54.5%
datafusion / vortex-compact / ns (0.543x ✅, 9↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
fineweb_q00/datafusion:vortex-compact 5843852 / 8431716 / -30.7% 🟢 23537513 / 36291471 / -35.1% 🟢 0.25 / 0.23 / +6.9%
fineweb_q01/datafusion:vortex-compact 49496930 / 100105330 / -50.6% 🟢 114554027 / 169865933 / -32.6% 🟢 0.43 / 0.59 / -26.7%
fineweb_q02/datafusion:vortex-compact 52556385 / 100805504 / -47.9% 🟢 112102848 / 163744948 / -31.5% 🟢 0.47 / 0.62 / -23.8%
fineweb_q03/datafusion:vortex-compact 264158238 / 531173219 / -50.3% 🟢 388354994 / 608794851 / -36.2% 🟢 0.68 / 0.87 / -22.0%
fineweb_q04/datafusion:vortex-compact 315065140 / 568634988 / -44.6% 🟢 458162919 / 679232313 / -32.5% 🟢 0.69 / 0.84 / -17.9%
fineweb_q05/datafusion:vortex-compact 266143712 / 514276812 / -48.2% 🟢 434951993 / 611832428 / -28.9% 🟢 0.61 / 0.84 / -27.2%
fineweb_q06/datafusion:vortex-compact 142932444 / 289054490 / -50.6% 🟢 322114538 / 418736275 / -23.1% 🟢 0.44 / 0.69 / -35.7%
fineweb_q07/datafusion:vortex-compact 146754994 / 296538121 / -50.5% 🟢 302296762 / 430419993 / -29.8% 🟢 0.49 / 0.69 / -29.5%
fineweb_q08/datafusion:vortex-compact 10628960 / 16245558 / -34.6% 🟢 79981009 / 64447680 / +24.1% 🔴 0.13 / 0.25 / -47.3%
datafusion / parquet / ns (0.630x ✅, 8↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
fineweb_q00/datafusion:parquet 7766962 / 8349684 / -7.0% 38080709 / 50248070 / -24.2% 🟢 0.20 / 0.17 / +22.7%
fineweb_q01/datafusion:parquet 113995579 / 191208254 / -40.4% 🟢 349999585 / 274920370 / +27.3% 🔴 0.33 / 0.70 / -53.2%
fineweb_q02/datafusion:parquet 111543954 / 186687712 / -40.3% 🟢 338988127 / 282597770 / +20.0% 🔴 0.33 / 0.66 / -50.2%
fineweb_q03/datafusion:parquet 113250510 / 188765412 / -40.0% 🟢 341500567 / 278704144 / +22.5% 🔴 0.33 / 0.68 / -51.0%
fineweb_q04/datafusion:parquet 125635404 / 208286109 / -39.7% 🟢 339819394 / 301703612 / +12.6% 🔴 0.37 / 0.69 / -46.4%
fineweb_q05/datafusion:parquet 124323741 / 204948502 / -39.3% 🟢 344515016 / 298275732 / +15.5% 🔴 0.36 / 0.69 / -47.5%
fineweb_q06/datafusion:parquet 112882655 / 189410080 / -40.4% 🟢 343616096 / 279350392 / +23.0% 🔴 0.33 / 0.68 / -51.5%
fineweb_q07/datafusion:parquet 112674692 / 189266876 / -40.5% 🟢 338412427 / 274647315 / +23.2% 🔴 0.33 / 0.69 / -51.7%
fineweb_q08/datafusion:parquet 113628915 / 187713936 / -39.5% 🟢 338120738 / 278341657 / +21.5% 🔴 0.34 / 0.67 / -50.2%
duckdb / vortex-file-compressed / ns (0.457x ✅, 9↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
fineweb_q00/duckdb:vortex-file-compressed 2471099 / 3395134 / -27.2% 🟢 5242280 / 15966548 / -67.2% 🟢 0.47 / 0.21 / +121.7%
fineweb_q01/duckdb:vortex-file-compressed 16403186 / 37812417 / -56.6% 🟢 72186317 / 131429296 / -45.1% 🟢 0.23 / 0.29 / -21.0%
fineweb_q02/duckdb:vortex-file-compressed 18727660 / 43314548 / -56.8% 🟢 75268837 / 124427587 / -39.5% 🟢 0.25 / 0.35 / -28.5%
fineweb_q03/duckdb:vortex-file-compressed 27394068 / 67325544 / -59.3% 🟢 231079751 / 184548680 / +25.2% 🔴 0.12 / 0.36 / -67.5%
fineweb_q04/duckdb:vortex-file-compressed 115397656 / 267333002 / -56.8% 🟢 266592676 / 387970563 / -31.3% 🟢 0.43 / 0.69 / -37.2%
fineweb_q05/duckdb:vortex-file-compressed 88038451 / 220527167 / -60.1% 🟢 241812929 / 331258604 / -27.0% 🟢 0.36 / 0.67 / -45.3%
fineweb_q06/duckdb:vortex-file-compressed 27641886 / 63339442 / -56.4% 🟢 239288101 / 253414617 / -5.6% 0.12 / 0.25 / -53.8%
fineweb_q07/duckdb:vortex-file-compressed 26045878 / 57206098 / -54.5% 🟢 234434982 / 163205626 / +43.6% 🔴 0.11 / 0.35 / -68.3%
fineweb_q08/duckdb:vortex-file-compressed 12332868 / 27170711 / -54.6% 🟢 109535769 / 89142739 / +22.9% 🔴 0.11 / 0.30 / -63.1%
duckdb / vortex-compact / ns (0.594x ✅, 9↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
fineweb_q00/duckdb:vortex-compact 2789821 / 3950838 / -29.4% 🟢 5440734 / 16578835 / -67.2% 🟢 0.51 / 0.24 / +115.2%
fineweb_q01/duckdb:vortex-compact 49343737 / 102913645 / -52.1% 🟢 103826886 / 192011960 / -45.9% 🟢 0.48 / 0.54 / -11.3%
fineweb_q02/duckdb:vortex-compact 51468314 / 108321974 / -52.5% 🟢 107807812 / 171162088 / -37.0% 🟢 0.48 / 0.63 / -24.6%
fineweb_q03/duckdb:vortex-compact 294435630 / 435364263 / -32.4% 🟢 380399437 / 558810299 / -31.9% 🟢 0.77 / 0.78 / -0.7%
fineweb_q04/duckdb:vortex-compact 311037122 / 454838498 / -31.6% 🟢 429771987 / 579599630 / -25.9% 🟢 0.72 / 0.78 / -7.8%
fineweb_q05/duckdb:vortex-compact 291488698 / 401476362 / -27.4% 🟢 379322972 / 515708625 / -26.4% 🟢 0.77 / 0.78 / -1.3%
fineweb_q06/duckdb:vortex-compact 146216695 / 238159000 / -38.6% 🟢 304105332 / 435150139 / -30.1% 🟢 0.48 / 0.55 / -12.1%
fineweb_q07/duckdb:vortex-compact 150260961 / 243979192 / -38.4% 🟢 290985457 / 359406001 / -19.0% 🟢 0.52 / 0.68 / -23.9%
fineweb_q08/duckdb:vortex-compact 11183750 / 24972434 / -55.2% 🟢 95542307 / 104423518 / -8.5% 0.12 / 0.24 / -51.1%
duckdb / parquet / ns (0.809x ✅, 7↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
fineweb_q00/duckdb:parquet 29563686 / 36594066 / -19.2% 🟢 30546740 / 36840273 / -17.1% 🟢 0.97 / 0.99 / -2.6%
fineweb_q01/duckdb:parquet 57728058 / 77872732 / -25.9% 🟢 87920130 / 105805755 / -16.9% 🟢 0.66 / 0.74 / -10.8%
fineweb_q02/duckdb:parquet 57058374 / 76403498 / -25.3% 🟢 88764868 / 104778560 / -15.3% 🟢 0.64 / 0.73 / -11.8%
fineweb_q03/duckdb:parquet 142852296 / 205495207 / -30.5% 🟢 339903292 / 422818207 / -19.6% 🟢 0.42 / 0.49 / -13.5%
fineweb_q04/duckdb:parquet 234081698 / 231643815 / +1.1% 346688074 / 444505187 / -22.0% 🟢 0.68 / 0.52 / +29.6%
fineweb_q05/duckdb:parquet 223277590 / 203706140 / +9.6% 342597745 / 425664995 / -19.5% 🟢 0.65 / 0.48 / +36.2%
fineweb_q06/duckdb:parquet 101464270 / 141662990 / -28.4% 🟢 278066124 / 273513801 / +1.7% 0.36 / 0.52 / -29.5%
fineweb_q07/duckdb:parquet 104303716 / 144213858 / -27.7% 🟢 270893205 / 324132565 / -16.4% 🟢 0.39 / 0.44 / -13.5%
fineweb_q08/duckdb:parquet 31373606 / 37804670 / -17.0% 🟢 35456992 / 38607511 / -8.2% 0.88 / 0.98 / -9.6%
File Size Changes (1 files changed, +0.0% overall, 1↑ 0↓)
File Scale Format Base HEAD Change %
sample.vortex 1.0 vortex-file-compressed 1.42 GB 1.42 GB +105.45 KB +0.0%

Totals:

  • vortex-compact: 1.23 GB → 1.23 GB (0.0%)
  • vortex-file-compressed: 1.42 GB → 1.42 GB (+0.0%)

@github-actions

github-actions Bot commented Sep 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-H SF=1 on S3 📖

Commits: PR dd8b5948 vs base 792cef74
Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: +10.8%
Engines: DataFusion No clear signal (+12.4%, environment too noisy confidence) · DuckDB No clear signal (+9.2%, environment too noisy confidence)
Vortex (geomean): hot 1.085x ➖ · cold 1.026x ➖
Parquet (geomean): hot 0.979x ➖ · cold 0.969x ➖
Shifts: Parquet (control) -2.1% · Median polish +4.4%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Table cells: Each cell reads PR / base / %diff. The hot and cold columns mark a change as 🔴 slower or 🟢 faster once it clears this suite's threshold; anything smaller is noise here and is left unmarked, as is hot/cold, because a shift in the warm-up ratio is not a win or a loss by itself.
datafusion / vortex-file-compressed / ns (1.005x ➖, 1↑ 3↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/datafusion:vortex-file-compressed 214236807 / 236815662 / -9.5% 650694870 / 701098189 / -7.2% 0.33 / 0.34 / -2.5%
tpch_q02/datafusion:vortex-file-compressed 336046678 / 351687306 / -4.4% 613448346 / 620850751 / -1.2% 0.55 / 0.57 / -3.3%
tpch_q03/datafusion:vortex-file-compressed 331196953 / 339536717 / -2.5% 1107742886 / 921036546 / +20.3% 0.30 / 0.37 / -18.9%
tpch_q04/datafusion:vortex-file-compressed 168402484 / 160551520 / +4.9% 441521432 / 363020333 / +21.6% 0.38 / 0.44 / -13.8%
tpch_q05/datafusion:vortex-file-compressed 365192043 / 346561475 / +5.4% 989905838 / 594920046 / +66.4% 🔴 0.37 / 0.58 / -36.7%
tpch_q06/datafusion:vortex-file-compressed 304475049 / 239946512 / +26.9% 427803649 / 431551774 / -0.9% 0.71 / 0.56 / +28.0%
tpch_q07/datafusion:vortex-file-compressed 479710140 / 336990129 / +42.4% 🔴 527811569 / 511417848 / +3.2% 0.91 / 0.66 / +37.9%
tpch_q08/datafusion:vortex-file-compressed 398763907 / 551069781 / -27.6% 678171488 / 815609334 / -16.9% 0.59 / 0.68 / -13.0%
tpch_q09/datafusion:vortex-file-compressed 479301866 / 286263798 / +67.4% 🔴 602468426 / 913670459 / -34.1% 🟢 0.80 / 0.31 / +153.9%
tpch_q10/datafusion:vortex-file-compressed 313656910 / 306724015 / +2.3% 472368019 / 540860313 / -12.7% 0.66 / 0.57 / +17.1%
tpch_q11/datafusion:vortex-file-compressed 286321584 / 289190458 / -1.0% 433031989 / 434383797 / -0.3% 0.66 / 0.67 / -0.7%
tpch_q12/datafusion:vortex-file-compressed 350490475 / 482797963 / -27.4% 571528541 / 634466936 / -9.9% 0.61 / 0.76 / -19.4%
tpch_q13/datafusion:vortex-file-compressed 117174939 / 122445662 / -4.3% 388523684 / 539338984 / -28.0% 0.30 / 0.23 / +32.8%
tpch_q14/datafusion:vortex-file-compressed 164162317 / 187240514 / -12.3% 324274086 / 346699503 / -6.5% 0.51 / 0.54 / -6.3%
tpch_q15/datafusion:vortex-file-compressed 285517111 / 321423436 / -11.2% 482060871 / 603729759 / -20.2% 0.59 / 0.53 / +11.2%
tpch_q16/datafusion:vortex-file-compressed 158561214 / 168224151 / -5.7% 323758528 / 332650949 / -2.7% 0.49 / 0.51 / -3.2%
tpch_q17/datafusion:vortex-file-compressed 275854276 / 279906572 / -1.4% 473159689 / 503500730 / -6.0% 0.58 / 0.56 / +4.9%
tpch_q18/datafusion:vortex-file-compressed 310332988 / 236897431 / +31.0% 🔴 952349699 / 457711324 / +108.1% 🔴 0.33 / 0.52 / -37.0%
tpch_q19/datafusion:vortex-file-compressed 395584516 / 369641830 / +7.0% 606493834 / 611784183 / -0.9% 0.65 / 0.60 / +8.0%
tpch_q20/datafusion:vortex-file-compressed 289694196 / 288518580 / +0.4% 508189181 / 432070180 / +17.6% 0.57 / 0.67 / -14.6%
tpch_q21/datafusion:vortex-file-compressed 465922512 / 420188971 / +10.9% 692479740 / 629783951 / +10.0% 0.67 / 0.67 / +0.8%
tpch_q22/datafusion:vortex-file-compressed 184686856 / 265727320 / -30.5% 🟢 361041384 / 521482225 / -30.8% 🟢 0.51 / 0.51 / +0.4%
datafusion / vortex-compact / ns (1.093x ➖, 0↑ 3↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/datafusion:vortex-compact 217788331 / 246362515 / -11.6% 610461348 / 589816110 / +3.5% 0.36 / 0.42 / -14.6%
tpch_q02/datafusion:vortex-compact 395535043 / 303511322 / +30.3% 🔴 787536577 / 624897956 / +26.0% 0.50 / 0.49 / +3.4%
tpch_q03/datafusion:vortex-compact 317429368 / 332826528 / -4.6% 736351886 / 848076720 / -13.2% 0.43 / 0.39 / +9.8%
tpch_q04/datafusion:vortex-compact 158193762 / 170817741 / -7.4% 345867206 / 326909470 / +5.8% 0.46 / 0.52 / -12.5%
tpch_q05/datafusion:vortex-compact 344759635 / 305906628 / +12.7% 666833470 / 559721376 / +19.1% 0.52 / 0.55 / -5.4%
tpch_q06/datafusion:vortex-compact 267716118 / 252749196 / +5.9% 412980062 / 418729220 / -1.4% 0.65 / 0.60 / +7.4%
tpch_q07/datafusion:vortex-compact 331548929 / 427419044 / -22.4% 505671274 / 564695144 / -10.5% 0.66 / 0.76 / -13.4%
tpch_q08/datafusion:vortex-compact 448655407 / 364310094 / +23.2% 601034500 / 547136425 / +9.9% 0.75 / 0.67 / +12.1%
tpch_q09/datafusion:vortex-compact 282794863 / 266798177 / +6.0% 479357517 / 486478027 / -1.5% 0.59 / 0.55 / +7.6%
tpch_q10/datafusion:vortex-compact 309724657 / 320883327 / -3.5% 452900527 / 580140197 / -21.9% 0.68 / 0.55 / +23.6%
tpch_q11/datafusion:vortex-compact 302182694 / 283362994 / +6.6% 419068635 / 457493884 / -8.4% 0.72 / 0.62 / +16.4%
tpch_q12/datafusion:vortex-compact 367942743 / 322747169 / +14.0% 583153502 / 512511555 / +13.8% 0.63 / 0.63 / +0.2%
tpch_q13/datafusion:vortex-compact 290632329 / 116228602 / +150.1% 🔴 476513613 / 527694844 / -9.7% 0.61 / 0.22 / +176.9%
tpch_q14/datafusion:vortex-compact 279364458 / 189464178 / +47.4% 🔴 478226167 / 335999324 / +42.3% 🔴 0.58 / 0.56 / +3.6%
tpch_q15/datafusion:vortex-compact 315191698 / 296270903 / +6.4% 457032064 / 507506462 / -9.9% 0.69 / 0.58 / +18.1%
tpch_q16/datafusion:vortex-compact 159778627 / 151773734 / +5.3% 323384856 / 319134333 / +1.3% 0.49 / 0.48 / +3.9%
tpch_q17/datafusion:vortex-compact 298335361 / 278975472 / +6.9% 426037516 / 471619370 / -9.7% 0.70 / 0.59 / +18.4%
tpch_q18/datafusion:vortex-compact 294958540 / 247099681 / +19.4% 427929674 / 437552868 / -2.2% 0.69 / 0.56 / +22.1%
tpch_q19/datafusion:vortex-compact 490787306 / 427485655 / +14.8% 576668566 / 710970296 / -18.9% 0.85 / 0.60 / +41.5%
tpch_q20/datafusion:vortex-compact 248393257 / 273474845 / -9.2% 433288694 / 459919464 / -5.8% 0.57 / 0.59 / -3.6%
tpch_q21/datafusion:vortex-compact 425167822 / 452806309 / -6.1% 560879763 / 615759209 / -8.9% 0.76 / 0.74 / +3.1%
tpch_q22/datafusion:vortex-compact 160906978 / 169295677 / -5.0% 351290175 / 356064817 / -1.3% 0.46 / 0.48 / -3.7%
datafusion / parquet / ns (0.933x ➖, 1↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/datafusion:parquet 177446788 / 222418594 / -20.2% 574319477 / 675564213 / -15.0% 0.31 / 0.33 / -6.2%
tpch_q02/datafusion:parquet 276690439 / 289433254 / -4.4% 471777758 / 541425488 / -12.9% 0.59 / 0.53 / +9.7%
tpch_q03/datafusion:parquet 278456942 / 276921846 / +0.6% 861867250 / 838137856 / +2.8% 0.32 / 0.33 / -2.2%
tpch_q04/datafusion:parquet 148229988 / 304030351 / -51.2% 🟢 363191047 / 459450488 / -21.0% 0.41 / 0.66 / -38.3%
tpch_q05/datafusion:parquet 377659066 / 369371606 / +2.2% 497815389 / 532255825 / -6.5% 0.76 / 0.69 / +9.3%
tpch_q06/datafusion:parquet 139399108 / 134916591 / +3.3% 336017964 / 330576907 / +1.6% 0.41 / 0.41 / +1.6%
tpch_q07/datafusion:parquet 353690095 / 359248419 / -1.5% 612761187 / 533792698 / +14.8% 0.58 / 0.67 / -14.2%
tpch_q08/datafusion:parquet 522012088 / 559203327 / -6.7% 600608872 / 600777947 / -0.0% 0.87 / 0.93 / -6.6%
tpch_q09/datafusion:parquet 419930975 / 458575061 / -8.4% 630768491 / 701991057 / -10.1% 0.67 / 0.65 / +1.9%
tpch_q10/datafusion:parquet 397294390 / 408384180 / -2.7% 532242258 / 551368707 / -3.5% 0.75 / 0.74 / +0.8%
tpch_q11/datafusion:parquet 263640372 / 286114067 / -7.9% 434837563 / 550687454 / -21.0% 0.61 / 0.52 / +16.7%
tpch_q12/datafusion:parquet 184968516 / 191721860 / -3.5% 471094615 / 469106476 / +0.4% 0.39 / 0.41 / -3.9%
tpch_q13/datafusion:parquet 420782829 / 451678915 / -6.8% 669821337 / 697504005 / -4.0% 0.63 / 0.65 / -3.0%
tpch_q14/datafusion:parquet 169059957 / 176556793 / -4.2% 311685813 / 328597229 / -5.1% 0.54 / 0.54 / +0.9%
tpch_q15/datafusion:parquet 294057771 / 274299214 / +7.2% 557826530 / 469601259 / +18.8% 0.53 / 0.58 / -9.8%
tpch_q16/datafusion:parquet 132040912 / 121320015 / +8.8% 301187481 / 349137025 / -13.7% 0.44 / 0.35 / +26.2%
tpch_q17/datafusion:parquet 307908903 / 296043367 / +4.0% 463085869 / 435794532 / +6.3% 0.66 / 0.68 / -2.1%
tpch_q18/datafusion:parquet 453379951 / 471211812 / -3.8% 664505313 / 612918823 / +8.4% 0.68 / 0.77 / -11.3%
tpch_q19/datafusion:parquet 185149437 / 198500821 / -6.7% 300591682 / 425331862 / -29.3% 0.62 / 0.47 / +32.0%
tpch_q20/datafusion:parquet 240475496 / 296160722 / -18.8% 457103750 / 470106740 / -2.8% 0.53 / 0.63 / -16.5%
tpch_q21/datafusion:parquet 356289552 / 363902843 / -2.1% 427759602 / 528406128 / -19.0% 0.83 / 0.69 / +20.9%
tpch_q22/datafusion:parquet 205670488 / 212284331 / -3.1% 490019840 / 417563917 / +17.4% 0.42 / 0.51 / -17.4%
duckdb / vortex-file-compressed / ns (1.118x ➖, 0↑ 3↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/duckdb:vortex-file-compressed 240277038 / 230949865 / +4.0% 374791100 / 260227389 / +44.0% 🔴 0.64 / 0.89 / -27.8%
tpch_q02/duckdb:vortex-file-compressed 448648585 / 410902608 / +9.2% 472308465 / 447383054 / +5.6% 0.95 / 0.92 / +3.4%
tpch_q03/duckdb:vortex-file-compressed 483019375 / 430462010 / +12.2% 497788780 / 542172879 / -8.2% 0.97 / 0.79 / +22.2%
tpch_q04/duckdb:vortex-file-compressed 316384230 / 252070930 / +25.5% 356894982 / 301845230 / +18.2% 0.89 / 0.84 / +6.2%
tpch_q05/duckdb:vortex-file-compressed 454910962 / 357868050 / +27.1% 521266490 / 434131307 / +20.1% 0.87 / 0.82 / +5.9%
tpch_q06/duckdb:vortex-file-compressed 368512212 / 245346966 / +50.2% 🔴 353784129 / 322807333 / +9.6% 1.04 / 0.76 / +37.0%
tpch_q07/duckdb:vortex-file-compressed 494978007 / 605229088 / -18.2% 559904269 / 629833417 / -11.1% 0.88 / 0.96 / -8.0%
tpch_q08/duckdb:vortex-file-compressed 600835980 / 547520962 / +9.7% 612146257 / 634138049 / -3.5% 0.98 / 0.86 / +13.7%
tpch_q09/duckdb:vortex-file-compressed 535536910 / 463597562 / +15.5% 697667285 / 548012350 / +27.3% 0.77 / 0.85 / -9.3%
tpch_q10/duckdb:vortex-file-compressed 431972048 / 426461209 / +1.3% 540490395 / 512397347 / +5.5% 0.80 / 0.83 / -4.0%
tpch_q11/duckdb:vortex-file-compressed 173785261 / 140328054 / +23.8% 161176109 / 191635904 / -15.9% 1.08 / 0.73 / +47.2%
tpch_q12/duckdb:vortex-file-compressed 607861366 / 687041822 / -11.5% 739000144 / 691715582 / +6.8% 0.82 / 0.99 / -17.2%
tpch_q13/duckdb:vortex-file-compressed 324904336 / 349522324 / -7.0% 352041987 / 368032234 / -4.3% 0.92 / 0.95 / -2.8%
tpch_q14/duckdb:vortex-file-compressed 461373446 / 225297671 / +104.8% 🔴 492621842 / 313264107 / +57.3% 🔴 0.94 / 0.72 / +30.2%
tpch_q15/duckdb:vortex-file-compressed 145127971 / 152753074 / -5.0% 221135266 / 232454993 / -4.9% 0.66 / 0.66 / -0.1%
tpch_q16/duckdb:vortex-file-compressed 184866183 / 201803940 / -8.4% 259874767 / 242594738 / +7.1% 0.71 / 0.83 / -14.5%
tpch_q17/duckdb:vortex-file-compressed 470958079 / 402706187 / +16.9% 513242274 / 453567624 / +13.2% 0.92 / 0.89 / +3.4%
tpch_q18/duckdb:vortex-file-compressed 332297700 / 314290871 / +5.7% 376549190 / 355552149 / +5.9% 0.88 / 0.88 / -0.2%
tpch_q19/duckdb:vortex-file-compressed 397602718 / 320435115 / +24.1% 458141599 / 847610129 / -45.9% 🟢 0.87 / 0.38 / +129.6%
tpch_q20/duckdb:vortex-file-compressed 535175845 / 377669758 / +41.7% 🔴 491747093 / 511539948 / -3.9% 1.09 / 0.74 / +47.4%
tpch_q21/duckdb:vortex-file-compressed 599225134 / 670788369 / -10.7% 663848554 / 723269185 / -8.2% 0.90 / 0.93 / -2.7%
tpch_q22/duckdb:vortex-file-compressed 120814186 / 116604399 / +3.6% 150547407 / 207674229 / -27.5% 0.80 / 0.56 / +42.9%
duckdb / vortex-compact / ns (1.126x ➖, 0↑ 4↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/duckdb:vortex-compact 262771807 / 227999849 / +15.3% 546275352 / 252912949 / +116.0% 🔴 0.48 / 0.90 / -46.6%
tpch_q02/duckdb:vortex-compact 376089001 / 367707785 / +2.3% 522076396 / 381614855 / +36.8% 🔴 0.72 / 0.96 / -25.2%
tpch_q03/duckdb:vortex-compact 585526780 / 403361996 / +45.2% 🔴 553476429 / 453502764 / +22.0% 1.06 / 0.89 / +18.9%
tpch_q04/duckdb:vortex-compact 260899919 / 250220334 / +4.3% 280982324 / 303670110 / -7.5% 0.93 / 0.82 / +12.7%
tpch_q05/duckdb:vortex-compact 413598844 / 385484929 / +7.3% 557925260 / 414559308 / +34.6% 🔴 0.74 / 0.93 / -20.3%
tpch_q06/duckdb:vortex-compact 306670051 / 254916765 / +20.3% 312503337 / 310450059 / +0.7% 0.98 / 0.82 / +19.5%
tpch_q07/duckdb:vortex-compact 655970764 / 437701369 / +49.9% 🔴 556254972 / 542647503 / +2.5% 1.18 / 0.81 / +46.2%
tpch_q08/duckdb:vortex-compact 503083187 / 530465656 / -5.2% 591667680 / 636166198 / -7.0% 0.85 / 0.83 / +2.0%
tpch_q09/duckdb:vortex-compact 459558048 / 479103094 / -4.1% 574069516 / 509193400 / +12.7% 0.80 / 0.94 / -14.9%
tpch_q10/duckdb:vortex-compact 377911624 / 405948121 / -6.9% 459780678 / 457475399 / +0.5% 0.82 / 0.89 / -7.4%
tpch_q11/duckdb:vortex-compact 132052191 / 137394145 / -3.9% 157152718 / 173022105 / -9.2% 0.84 / 0.79 / +5.8%
tpch_q12/duckdb:vortex-compact 508163604 / 356469006 / +42.6% 🔴 556333678 / 459891712 / +21.0% 0.91 / 0.78 / +17.8%
tpch_q13/duckdb:vortex-compact 264375845 / 226848245 / +16.5% 426010194 / 307322442 / +38.6% 🔴 0.62 / 0.74 / -15.9%
tpch_q14/duckdb:vortex-compact 299543343 / 235931792 / +27.0% 357953116 / 476350891 / -24.9% 0.84 / 0.50 / +69.0%
tpch_q15/duckdb:vortex-compact 224659429 / 162826993 / +38.0% 🔴 258898298 / 277614589 / -6.7% 0.87 / 0.59 / +47.9%
tpch_q16/duckdb:vortex-compact 174157081 / 184142772 / -5.4% 233081658 / 242594647 / -3.9% 0.75 / 0.76 / -1.6%
tpch_q17/duckdb:vortex-compact 473491705 / 388892138 / +21.8% 783978017 / 432168174 / +81.4% 🔴 0.60 / 0.90 / -32.9%
tpch_q18/duckdb:vortex-compact 360827267 / 288416345 / +25.1% 376469006 / 380031430 / -0.9% 0.96 / 0.76 / +26.3%
tpch_q19/duckdb:vortex-compact 403952922 / 358898789 / +12.6% 401819926 / 383552969 / +4.8% 1.01 / 0.94 / +7.4%
tpch_q20/duckdb:vortex-compact 474650613 / 367204091 / +29.3% 583740634 / 424664319 / +37.5% 🔴 0.81 / 0.86 / -6.0%
tpch_q21/duckdb:vortex-compact 755607356 / 930657058 / -18.8% 799987643 / 972964528 / -17.8% 0.94 / 0.96 / -1.3%
tpch_q22/duckdb:vortex-compact 113322899 / 115428160 / -1.8% 140706123 / 159090936 / -11.6% 0.81 / 0.73 / +11.0%
duckdb / parquet / ns (1.027x ➖, 0↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/duckdb:parquet 313796581 / 303396926 / +3.4% 320913256 / 348738501 / -8.0% 0.98 / 0.87 / +12.4%
tpch_q02/duckdb:parquet 696677067 / 710757944 / -2.0% 680938517 / 697211014 / -2.3% 1.02 / 1.02 / +0.4%
tpch_q03/duckdb:parquet 874741688 / 806839785 / +8.4% 914023947 / 848453194 / +7.7% 0.96 / 0.95 / +0.6%
tpch_q04/duckdb:parquet 573720273 / 498292752 / +15.1% 569781287 / 555924013 / +2.5% 1.01 / 0.90 / +12.3%
tpch_q05/duckdb:parquet 957476605 / 968750787 / -1.2% 1046634734 / 991351326 / +5.6% 0.91 / 0.98 / -6.4%
tpch_q06/duckdb:parquet 319862834 / 295333473 / +8.3% 254090134 / 327036412 / -22.3% 1.26 / 0.90 / +39.4%
tpch_q07/duckdb:parquet 936940298 / 873866916 / +7.2% 880912823 / 844058555 / +4.4% 1.06 / 1.04 / +2.7%
tpch_q08/duckdb:parquet 1126202545 / 1130691036 / -0.4% 1109389625 / 1162649792 / -4.6% 1.02 / 0.97 / +4.4%
tpch_q09/duckdb:parquet 1032647935 / 1052326966 / -1.9% 1173804479 / 975341688 / +20.3% 0.88 / 1.08 / -18.5%
tpch_q10/duckdb:parquet 1021716829 / 1008275203 / +1.3% 1058139693 / 1023372905 / +3.4% 0.97 / 0.99 / -2.0%
tpch_q11/duckdb:parquet 478666760 / 444076071 / +7.8% 477276696 / 463355381 / +3.0% 1.00 / 0.96 / +4.6%
tpch_q12/duckdb:parquet 566722577 / 523242683 / +8.3% 622467362 / 584589875 / +6.5% 0.91 / 0.90 / +1.7%
tpch_q13/duckdb:parquet 868007776 / 842578700 / +3.0% 1058760722 / 899766875 / +17.7% 0.82 / 0.94 / -12.5%
tpch_q14/duckdb:parquet 523526065 / 498216258 / +5.1% 521267779 / 518098441 / +0.6% 1.00 / 0.96 / +4.4%
tpch_q15/duckdb:parquet 378371848 / 407593927 / -7.2% 328171680 / 497857740 / -34.1% 🟢 1.15 / 0.82 / +40.8%
tpch_q16/duckdb:parquet 527801925 / 521950652 / +1.1% 545848569 / 624102211 / -12.5% 0.97 / 0.84 / +15.6%
tpch_q17/duckdb:parquet 529052711 / 493074466 / +7.3% 531468198 / 544167119 / -2.3% 1.00 / 0.91 / +9.9%
tpch_q18/duckdb:parquet 621324552 / 686673132 / -9.5% 743786104 / 698762514 / +6.4% 0.84 / 0.98 / -15.0%
tpch_q19/duckdb:parquet 619779135 / 554771290 / +11.7% 662935057 / 641430800 / +3.4% 0.93 / 0.86 / +8.1%
tpch_q20/duckdb:parquet 786916604 / 863994013 / -8.9% 895184218 / 839655912 / +6.6% 0.88 / 1.03 / -14.6%
tpch_q21/duckdb:parquet 773952018 / 757393962 / +2.2% 778196215 / 1003671331 / -22.5% 0.99 / 0.75 / +31.8%
tpch_q22/duckdb:parquet 448444195 / 425567994 / +5.4% 484890047 / 416578046 / +16.4% 0.92 / 1.02 / -9.5%

@github-actions

github-actions Bot commented Sep 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-DS SF=1 on NVME 📖

Commits: PR dd8b5948 vs base 792cef74
Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: -13.0%
Engines: DataFusion No clear signal (-3.3%, environment too noisy confidence) · DuckDB Likely improvement (-21.7%, environment too noisy confidence)
Vortex (geomean): hot 0.721x ✅ · cold 0.620x ✅
Parquet (geomean): hot 0.828x ✅ · cold 0.732x ✅
Shifts: Parquet (control) -17.2% · Median polish -24.3%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Table cells: Each cell reads PR / base / %diff. The hot and cold columns mark a change as 🔴 slower or 🟢 faster once it clears this suite's threshold; anything smaller is noise here and is left unmarked, as is hot/cold, because a shift in the warm-up ratio is not a win or a loss by itself.
datafusion / vortex-file-compressed / ns (0.774x ✅, 98↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpcds_q01/datafusion:vortex-file-compressed 22481102 / 29725591 / -24.4% 🟢 52599783 / 80334839 / -34.5% 🟢 0.43 / 0.37 / +15.5%
tpcds_q02/datafusion:vortex-file-compressed 39302378 / 50898544 / -22.8% 🟢 68778544 / 103000460 / -33.2% 🟢 0.57 / 0.49 / +15.6%
tpcds_q03/datafusion:vortex-file-compressed 10997374 / 15032395 / -26.8% 🟢 37607815 / 60087583 / -37.4% 🟢 0.29 / 0.25 / +16.9%
tpcds_q04/datafusion:vortex-file-compressed 138542378 / 193361345 / -28.4% 🟢 180208008 / 270367699 / -33.3% 🟢 0.77 / 0.72 / +7.5%
tpcds_q05/datafusion:vortex-file-compressed 49550248 / 62284902 / -20.4% 🟢 81016990 / 126528969 / -36.0% 🟢 0.61 / 0.49 / +24.2%
tpcds_q06/datafusion:vortex-file-compressed 22152938 / 26492492 / -16.4% 🟢 52011342 / 82049900 / -36.6% 🟢 0.43 / 0.32 / +31.9%
tpcds_q07/datafusion:vortex-file-compressed 30445213 / 42198112 / -27.9% 🟢 69233311 / 110536482 / -37.4% 🟢 0.44 / 0.38 / +15.2%
tpcds_q08/datafusion:vortex-file-compressed 25137552 / 29474975 / -14.7% 🟢 53944972 / 84067582 / -35.8% 🟢 0.47 / 0.35 / +32.9%
tpcds_q09/datafusion:vortex-file-compressed 20157220 / 25261890 / -20.2% 🟢 47328839 / 69741837 / -32.1% 🟢 0.43 / 0.36 / +17.6%
tpcds_q10/datafusion:vortex-file-compressed 31380239 / 37725123 / -16.8% 🟢 63358095 / 92740842 / -31.7% 🟢 0.50 / 0.41 / +21.8%
tpcds_q11/datafusion:vortex-file-compressed 72301659 / 108551871 / -33.4% 🟢 105344445 / 172579932 / -39.0% 🟢 0.69 / 0.63 / +9.1%
tpcds_q12/datafusion:vortex-file-compressed 14799080 / 20024108 / -26.1% 🟢 40686137 / 67350516 / -39.6% 🟢 0.36 / 0.30 / +22.3%
tpcds_q13/datafusion:vortex-file-compressed 34642640 / 40828130 / -15.2% 🟢 69302578 / 94094996 / -26.3% 🟢 0.50 / 0.43 / +15.2%
tpcds_q14/datafusion:vortex-file-compressed 107478475 / 140201920 / -23.3% 🟢 144073878 / 206294530 / -30.2% 🟢 0.75 / 0.68 / +9.8%
tpcds_q15/datafusion:vortex-file-compressed 21747820 / 30073392 / -27.7% 🟢 52767113 / 83369809 / -36.7% 🟢 0.41 / 0.36 / +14.3%
tpcds_q16/datafusion:vortex-file-compressed 20416669 / 26719631 / -23.6% 🟢 51514927 / 77246709 / -33.3% 🟢 0.40 / 0.35 / +14.6%
tpcds_q17/datafusion:vortex-file-compressed 52456249 / 69563710 / -24.6% 🟢 83623665 / 135505219 / -38.3% 🟢 0.63 / 0.51 / +22.2%
tpcds_q18/datafusion:vortex-file-compressed 70951430 / 81121088 / -12.5% 🟢 101605268 / 160458493 / -36.7% 🟢 0.70 / 0.51 / +38.1%
tpcds_q19/datafusion:vortex-file-compressed 19083476 / 24069269 / -20.7% 🟢 51329142 / 77139913 / -33.5% 🟢 0.37 / 0.31 / +19.2%
tpcds_q20/datafusion:vortex-file-compressed 16403668 / 24394868 / -32.8% 🟢 42363787 / 73230887 / -42.2% 🟢 0.39 / 0.33 / +16.2%
tpcds_q21/datafusion:vortex-file-compressed 20171976 / 28549633 / -29.3% 🟢 53879971 / 88538675 / -39.1% 🟢 0.37 / 0.32 / +16.1%
tpcds_q22/datafusion:vortex-file-compressed 76165760 / 115114028 / -33.8% 🟢 108954368 / 178665398 / -39.0% 🟢 0.70 / 0.64 / +8.5%
tpcds_q23/datafusion:vortex-file-compressed 87132044 / 130428042 / -33.2% 🟢 120351799 / 201986391 / -40.4% 🟢 0.72 / 0.65 / +12.1%
tpcds_q24/datafusion:vortex-file-compressed 65746606 / 83439470 / -21.2% 🟢 98354463 / 152602761 / -35.5% 🟢 0.67 / 0.55 / +22.3%
tpcds_q25/datafusion:vortex-file-compressed 49774653 / 66598732 / -25.3% 🟢 85242346 / 144184315 / -40.9% 🟢 0.58 / 0.46 / +26.4%
tpcds_q26/datafusion:vortex-file-compressed 27462648 / 37687302 / -27.1% 🟢 59988483 / 91779791 / -34.6% 🟢 0.46 / 0.41 / +11.5%
tpcds_q27/datafusion:vortex-file-compressed 64657112 / 93420699 / -30.8% 🟢 111370679 / 168406286 / -33.9% 🟢 0.58 / 0.55 / +4.7%
tpcds_q28/datafusion:vortex-file-compressed 21531878 / 23992670 / -10.3% 🟢 49603969 / 70597634 / -29.7% 🟢 0.43 / 0.34 / +27.7%
tpcds_q29/datafusion:vortex-file-compressed 49806884 / 67920980 / -26.7% 🟢 87665584 / 141910843 / -38.2% 🟢 0.57 / 0.48 / +18.7%
tpcds_q30/datafusion:vortex-file-compressed 25559932 / 33479842 / -23.7% 🟢 53649974 / 87023543 / -38.4% 🟢 0.48 / 0.38 / +23.8%
tpcds_q31/datafusion:vortex-file-compressed 61034167 / 74346504 / -17.9% 🟢 96699674 / 143071339 / -32.4% 🟢 0.63 / 0.52 / +21.5%
tpcds_q32/datafusion:vortex-file-compressed 14007930 / 17703222 / -20.9% 🟢 40401330 / 61052051 / -33.8% 🟢 0.35 / 0.29 / +19.6%
tpcds_q33/datafusion:vortex-file-compressed 27071239 / 33592549 / -19.4% 🟢 58653657 / 90042117 / -34.9% 🟢 0.46 / 0.37 / +23.7%
tpcds_q34/datafusion:vortex-file-compressed 19612472 / 24923604 / -21.3% 🟢 51484465 / 77054180 / -33.2% 🟢 0.38 / 0.32 / +17.8%
tpcds_q35/datafusion:vortex-file-compressed 37987804 / 51194710 / -25.8% 🟢 70741171 / 106939320 / -33.8% 🟢 0.54 / 0.48 / +12.2%
tpcds_q36/datafusion:vortex-file-compressed 45369890 / 59564882 / -23.8% 🟢 81670238 / 133095019 / -38.6% 🟢 0.56 / 0.45 / +24.1%
tpcds_q37/datafusion:vortex-file-compressed 12673044 / 17255730 / -26.6% 🟢 49535999 / 67684301 / -26.8% 🟢 0.26 / 0.25 / +0.3%
tpcds_q38/datafusion:vortex-file-compressed 30500760 / 38731400 / -21.3% 🟢 62045003 / 91259150 / -32.0% 🟢 0.49 / 0.42 / +15.8%
tpcds_q39/datafusion:vortex-file-compressed 82602422 / 94813761 / -12.9% 🟢 116043732 / 152095392 / -23.7% 🟢 0.71 / 0.62 / +14.2%
tpcds_q40/datafusion:vortex-file-compressed 22585668 / 31178573 / -27.6% 🟢 52729855 / 90860276 / -42.0% 🟢 0.43 / 0.34 / +24.8%
tpcds_q41/datafusion:vortex-file-compressed 15335663 / 18813688 / -18.5% 🟢 38829133 / 59542575 / -34.8% 🟢 0.39 / 0.32 / +25.0%
tpcds_q42/datafusion:vortex-file-compressed 10413181 / 13693868 / -24.0% 🟢 36335999 / 58757431 / -38.2% 🟢 0.29 / 0.23 / +23.0%
tpcds_q43/datafusion:vortex-file-compressed 13099052 / 17851176 / -26.6% 🟢 41612397 / 67040384 / -37.9% 🟢 0.31 / 0.27 / +18.2%
tpcds_q44/datafusion:vortex-file-compressed 26347466 / 32795410 / -19.7% 🟢 60095250 / 89916073 / -33.2% 🟢 0.44 / 0.36 / +20.2%
tpcds_q45/datafusion:vortex-file-compressed 25450228 / 32063804 / -20.6% 🟢 54397458 / 86715532 / -37.3% 🟢 0.47 / 0.37 / +26.5%
tpcds_q46/datafusion:vortex-file-compressed 25052189 / 32390990 / -22.7% 🟢 60186196 / 93478587 / -35.6% 🟢 0.42 / 0.35 / +20.1%
tpcds_q47/datafusion:vortex-file-compressed 85410010 / 116526812 / -26.7% 🟢 123959245 / 196183308 / -36.8% 🟢 0.69 / 0.59 / +16.0%
tpcds_q48/datafusion:vortex-file-compressed 29214646 / 32538212 / -10.2% 🟢 62090800 / 89374397 / -30.5% 🟢 0.47 / 0.36 / +29.2%
tpcds_q49/datafusion:vortex-file-compressed 52975866 / 64693212 / -18.1% 🟢 88669417 / 125616087 / -29.4% 🟢 0.60 / 0.52 / +16.0%
tpcds_q50/datafusion:vortex-file-compressed 34041386 / 44241434 / -23.1% 🟢 63365593 / 99291735 / -36.2% 🟢 0.54 / 0.45 / +20.6%
tpcds_q51/datafusion:vortex-file-compressed 61718618 / 71684813 / -13.9% 🟢 99838476 / 137307292 / -27.3% 🟢 0.62 / 0.52 / +18.4%
tpcds_q52/datafusion:vortex-file-compressed 11409708 / 14277214 / -20.1% 🟢 37736403 / 59350351 / -36.4% 🟢 0.30 / 0.24 / +25.7%
tpcds_q53/datafusion:vortex-file-compressed 17329593 / 21850184 / -20.7% 🟢 46626989 / 72086592 / -35.3% 🟢 0.37 / 0.30 / +22.6%
tpcds_q54/datafusion:vortex-file-compressed 31329776 / 36935378 / -15.2% 🟢 64715922 / 92046337 / -29.7% 🟢 0.48 / 0.40 / +20.6%
tpcds_q55/datafusion:vortex-file-compressed 10377010 / 14143113 / -26.6% 🟢 36124775 / 58534335 / -38.3% 🟢 0.29 / 0.24 / +18.9%
tpcds_q56/datafusion:vortex-file-compressed 26432206 / 32082754 / -17.6% 🟢 58295099 / 94909564 / -38.6% 🟢 0.45 / 0.34 / +34.1%
tpcds_q57/datafusion:vortex-file-compressed 76397866 / 93113214 / -18.0% 🟢 110290090 / 171816884 / -35.8% 🟢 0.69 / 0.54 / +27.8%
tpcds_q58/datafusion:vortex-file-compressed 40577988 / 54301881 / -25.3% 🟢 77929508 / 120798981 / -35.5% 🟢 0.52 / 0.45 / +15.8%
tpcds_q59/datafusion:vortex-file-compressed 35714853 / 52714802 / -32.2% 🟢 65022954 / 107741560 / -39.6% 🟢 0.55 / 0.49 / +12.3%
tpcds_q60/datafusion:vortex-file-compressed 26170020 / 32743882 / -20.1% 🟢 54876885 / 84684172 / -35.2% 🟢 0.48 / 0.39 / +23.3%
tpcds_q61/datafusion:vortex-file-compressed 27906592 / 39034022 / -28.5% 🟢 59781533 / 98195980 / -39.1% 🟢 0.47 / 0.40 / +17.4%
tpcds_q62/datafusion:vortex-file-compressed 17793470 / 23679512 / -24.9% 🟢 46197691 / 73672245 / -37.3% 🟢 0.39 / 0.32 / +19.8%
tpcds_q63/datafusion:vortex-file-compressed 17694495 / 22056618 / -19.8% 🟢 46913339 / 74141506 / -36.7% 🟢 0.38 / 0.30 / +26.8%
tpcds_q64/datafusion:vortex-file-compressed 289467716 / 365210196 / -20.7% 🟢 334466895 / 481165364 / -30.5% 🟢 0.87 / 0.76 / +14.0%
tpcds_q65/datafusion:vortex-file-compressed 61095914 / 65624564 / -6.9% 93272396 / 129514990 / -28.0% 🟢 0.66 / 0.51 / +29.3%
tpcds_q66/datafusion:vortex-file-compressed 60186675 / 71793447 / -16.2% 🟢 94356596 / 138795572 / -32.0% 🟢 0.64 / 0.52 / +23.3%
tpcds_q67/datafusion:vortex-file-compressed 82765795 / 109686252 / -24.5% 🟢 125948651 / 197058493 / -36.1% 🟢 0.66 / 0.56 / +18.1%
tpcds_q68/datafusion:vortex-file-compressed 24308902 / 31106142 / -21.9% 🟢 59972905 / 101457444 / -40.9% 🟢 0.41 / 0.31 / +32.2%
tpcds_q69/datafusion:vortex-file-compressed 30008276 / 38056830 / -21.1% 🟢 61345327 / 89202035 / -31.2% 🟢 0.49 / 0.43 / +14.7%
tpcds_q70/datafusion:vortex-file-compressed 76222286 / 108826342 / -30.0% 🟢 107591660 / 171806354 / -37.4% 🟢 0.71 / 0.63 / +11.8%
tpcds_q71/datafusion:vortex-file-compressed 20894978 / 24458812 / -14.6% 🟢 50805206 / 81698277 / -37.8% 🟢 0.41 / 0.30 / +37.4%
tpcds_q72/datafusion:vortex-file-compressed 958381594 / 1448095803 / -33.8% 🟢 977353954 / 1560943442 / -37.4% 🟢 0.98 / 0.93 / +5.7%
tpcds_q73/datafusion:vortex-file-compressed 18254388 / 24332816 / -25.0% 🟢 50409563 / 81094020 / -37.8% 🟢 0.36 / 0.30 / +20.7%
tpcds_q74/datafusion:vortex-file-compressed 49356182 / 70064732 / -29.6% 🟢 82174507 / 130828331 / -37.2% 🟢 0.60 / 0.54 / +12.2%
tpcds_q75/datafusion:vortex-file-compressed 108322470 / 133822072 / -19.1% 🟢 148080507 / 215689593 / -31.3% 🟢 0.73 / 0.62 / +17.9%
tpcds_q76/datafusion:vortex-file-compressed 30827650 / 37958649 / -18.8% 🟢 61452810 / 96193756 / -36.1% 🟢 0.50 / 0.39 / +27.1%
tpcds_q77/datafusion:vortex-file-compressed 34526070 / 41727915 / -17.3% 🟢 65738770 / 107301178 / -38.7% 🟢 0.53 / 0.39 / +35.1%
tpcds_q78/datafusion:vortex-file-compressed 74872234 / 106841621 / -29.9% 🟢 114755172 / 184497685 / -37.8% 🟢 0.65 / 0.58 / +12.7%
tpcds_q79/datafusion:vortex-file-compressed 21710254 / 26778659 / -18.9% 🟢 53236690 / 82011315 / -35.1% 🟢 0.41 / 0.33 / +24.9%
tpcds_q80/datafusion:vortex-file-compressed 74260188 / 96777830 / -23.3% 🟢 114034101 / 193615365 / -41.1% 🟢 0.65 / 0.50 / +30.3%
tpcds_q81/datafusion:vortex-file-compressed 25756994 / 35603198 / -27.7% 🟢 52954247 / 84203718 / -37.1% 🟢 0.49 / 0.42 / +15.0%
tpcds_q82/datafusion:vortex-file-compressed 13349252 / 17491660 / -23.7% 🟢 46757234 / 69860567 / -33.1% 🟢 0.29 / 0.25 / +14.0%
tpcds_q83/datafusion:vortex-file-compressed 36153853 / 45865178 / -21.2% 🟢 64879593 / 98939260 / -34.4% 🟢 0.56 / 0.46 / +20.2%
tpcds_q84/datafusion:vortex-file-compressed 12415901 / 15902404 / -21.9% 🟢 35077657 / 53468774 / -34.4% 🟢 0.35 / 0.30 / +19.0%
tpcds_q85/datafusion:vortex-file-compressed 55122344 / 63498582 / -13.2% 🟢 92195482 / 136143765 / -32.3% 🟢 0.60 / 0.47 / +28.2%
tpcds_q86/datafusion:vortex-file-compressed 15216422 / 19589071 / -22.3% 🟢 41488209 / 62185903 / -33.3% 🟢 0.37 / 0.32 / +16.4%
tpcds_q87/datafusion:vortex-file-compressed 29123210 / 38973076 / -25.3% 🟢 60778120 / 95358641 / -36.3% 🟢 0.48 / 0.41 / +17.2%
tpcds_q88/datafusion:vortex-file-compressed 42836871 / 50367776 / -15.0% 🟢 78277564 / 116302342 / -32.7% 🟢 0.55 / 0.43 / +26.4%
tpcds_q89/datafusion:vortex-file-compressed 20060159 / 25733456 / -22.0% 🟢 50254536 / 77662250 / -35.3% 🟢 0.40 / 0.33 / +20.5%
tpcds_q90/datafusion:vortex-file-compressed 12483687 / 14492588 / -13.9% 🟢 36971214 / 54242665 / -31.8% 🟢 0.34 / 0.27 / +26.4%
tpcds_q91/datafusion:vortex-file-compressed 19025554 / 22461810 / -15.3% 🟢 47198138 / 71175127 / -33.7% 🟢 0.40 / 0.32 / +27.7%
tpcds_q92/datafusion:vortex-file-compressed 14154122 / 17424226 / -18.8% 🟢 40309572 / 62799055 / -35.8% 🟢 0.35 / 0.28 / +26.6%
tpcds_q93/datafusion:vortex-file-compressed 21033964 / 30198057 / -30.3% 🟢 52339815 / 86103343 / -39.2% 🟢 0.40 / 0.35 / +14.6%
tpcds_q94/datafusion:vortex-file-compressed 18995312 / 24243276 / -21.6% 🟢 47382843 / 72361315 / -34.5% 🟢 0.40 / 0.34 / +19.7%
tpcds_q95/datafusion:vortex-file-compressed 40949484 / 53510701 / -23.5% 🟢 71790875 / 112293622 / -36.1% 🟢 0.57 / 0.48 / +19.7%
tpcds_q96/datafusion:vortex-file-compressed 9209222 / 11802814 / -22.0% 🟢 35077281 / 55474153 / -36.8% 🟢 0.26 / 0.21 / +23.4%
tpcds_q97/datafusion:vortex-file-compressed 20630127 / 27280985 / -24.4% 🟢 52950999 / 83887125 / -36.9% 🟢 0.39 / 0.33 / +19.8%
tpcds_q98/datafusion:vortex-file-compressed 15378905 / 22433524 / -31.4% 🟢 46465331 / 77419930 / -40.0% 🟢 0.33 / 0.29 / +14.2%
tpcds_q99/datafusion:vortex-file-compressed 21428274 / 27573132 / -22.3% 🟢 49582960 / 78457347 / -36.8% 🟢 0.43 / 0.35 / +23.0%
datafusion / vortex-compact / ns (0.738x ✅, 99↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpcds_q01/datafusion:vortex-compact 24611713 / 32039322 / -23.2% 🟢 54140070 / 87807621 / -38.3% 🟢 0.45 / 0.36 / +24.6%
tpcds_q02/datafusion:vortex-compact 43357613 / 58927086 / -26.4% 🟢 70493856 / 107424645 / -34.4% 🟢 0.62 / 0.55 / +12.1%
tpcds_q03/datafusion:vortex-compact 13612094 / 18649038 / -27.0% 🟢 40898140 / 67596788 / -39.5% 🟢 0.33 / 0.28 / +20.6%
tpcds_q04/datafusion:vortex-compact 144552304 / 223966339 / -35.5% 🟢 192913267 / 289881777 / -33.5% 🟢 0.75 / 0.77 / -3.0%
tpcds_q05/datafusion:vortex-compact 51591476 / 64677730 / -20.2% 🟢 87606604 / 132524368 / -33.9% 🟢 0.59 / 0.49 / +20.7%
tpcds_q06/datafusion:vortex-compact 23856851 / 30619416 / -22.1% 🟢 53380030 / 86719305 / -38.4% 🟢 0.45 / 0.35 / +26.6%
tpcds_q07/datafusion:vortex-compact 37850930 / 55465226 / -31.8% 🟢 70068173 / 112516067 / -37.7% 🟢 0.54 / 0.49 / +9.6%
tpcds_q08/datafusion:vortex-compact 30550002 / 35014104 / -12.7% 🟢 58675312 / 92891128 / -36.8% 🟢 0.52 / 0.38 / +38.1%
tpcds_q09/datafusion:vortex-compact 27801331 / 43730098 / -36.4% 🟢 52703046 / 90448212 / -41.7% 🟢 0.53 / 0.48 / +9.1%
tpcds_q10/datafusion:vortex-compact 42890330 / 56372852 / -23.9% 🟢 73580519 / 110001216 / -33.1% 🟢 0.58 / 0.51 / +13.7%
tpcds_q11/datafusion:vortex-compact 79901995 / 119373077 / -33.1% 🟢 109953693 / 187930809 / -41.5% 🟢 0.73 / 0.64 / +14.4%
tpcds_q12/datafusion:vortex-compact 20142496 / 26616022 / -24.3% 🟢 48234153 / 77134929 / -37.5% 🟢 0.42 / 0.35 / +21.0%
tpcds_q13/datafusion:vortex-compact 63353712 / 125891009 / -49.7% 🟢 94619993 / 192784195 / -50.9% 🟢 0.67 / 0.65 / +2.5%
tpcds_q14/datafusion:vortex-compact 112042726 / 153008959 / -26.8% 🟢 147410068 / 223631710 / -34.1% 🟢 0.76 / 0.68 / +11.1%
tpcds_q15/datafusion:vortex-compact 25079690 / 33039166 / -24.1% 🟢 52694785 / 83641144 / -37.0% 🟢 0.48 / 0.40 / +20.5%
tpcds_q16/datafusion:vortex-compact 23970763 / 33989498 / -29.5% 🟢 53662651 / 80154844 / -33.1% 🟢 0.45 / 0.42 / +5.3%
tpcds_q17/datafusion:vortex-compact 60478136 / 79450222 / -23.9% 🟢 93595233 / 152499405 / -38.6% 🟢 0.65 / 0.52 / +24.0%
tpcds_q18/datafusion:vortex-compact 76888650 / 92396928 / -16.8% 🟢 104185445 / 150677855 / -30.9% 🟢 0.74 / 0.61 / +20.4%
tpcds_q19/datafusion:vortex-compact 24852749 / 32857802 / -24.4% 🟢 56892803 / 86348089 / -34.1% 🟢 0.44 / 0.38 / +14.8%
tpcds_q20/datafusion:vortex-compact 21551000 / 30615238 / -29.6% 🟢 51934864 / 83491591 / -37.8% 🟢 0.41 / 0.37 / +13.2%
tpcds_q21/datafusion:vortex-compact 22936024 / 32089796 / -28.5% 🟢 53679537 / 93581080 / -42.6% 🟢 0.43 / 0.34 / +24.6%
tpcds_q22/datafusion:vortex-compact 76938748 / 128583820 / -40.2% 🟢 111406618 / 140962197 / -21.0% 🟢 0.69 / 0.91 / -24.3%
tpcds_q23/datafusion:vortex-compact 93364497 / 142570078 / -34.5% 🟢 128265069 / 205372498 / -37.5% 🟢 0.73 / 0.69 / +4.9%
tpcds_q24/datafusion:vortex-compact 74223718 / 96332526 / -23.0% 🟢 108385348 / 162828797 / -33.4% 🟢 0.68 / 0.59 / +15.8%
tpcds_q25/datafusion:vortex-compact 53074607 / 78341934 / -32.3% 🟢 86437124 / 154693394 / -44.1% 🟢 0.61 / 0.51 / +21.2%
tpcds_q26/datafusion:vortex-compact 33288764 / 48181457 / -30.9% 🟢 64442296 / 95352301 / -32.4% 🟢 0.52 / 0.51 / +2.2%
tpcds_q27/datafusion:vortex-compact 76389294 / 111738520 / -31.6% 🟢 115209399 / 180263773 / -36.1% 🟢 0.66 / 0.62 / +7.0%
tpcds_q28/datafusion:vortex-compact 36163874 / 65366723 / -44.7% 🟢 65927708 / 116435937 / -43.4% 🟢 0.55 / 0.56 / -2.3%
tpcds_q29/datafusion:vortex-compact 58273600 / 76253180 / -23.6% 🟢 94164791 / 146664116 / -35.8% 🟢 0.62 / 0.52 / +19.0%
tpcds_q30/datafusion:vortex-compact 29265470 / 38622652 / -24.2% 🟢 56886520 / 84712878 / -32.8% 🟢 0.51 / 0.46 / +12.8%
tpcds_q31/datafusion:vortex-compact 76459519 / 98493062 / -22.4% 🟢 108493309 / 167859236 / -35.4% 🟢 0.70 / 0.59 / +20.1%
tpcds_q32/datafusion:vortex-compact 16185535 / 22541080 / -28.2% 🟢 45505286 / 72696822 / -37.4% 🟢 0.36 / 0.31 / +14.7%
tpcds_q33/datafusion:vortex-compact 29997100 / 38117417 / -21.3% 🟢 63518442 / 100932285 / -37.1% 🟢 0.47 / 0.38 / +25.1%
tpcds_q34/datafusion:vortex-compact 24214609 / 34830300 / -30.5% 🟢 59300409 / 95447383 / -37.9% 🟢 0.41 / 0.36 / +11.9%
tpcds_q35/datafusion:vortex-compact 46331478 / 64663463 / -28.3% 🟢 78931684 / 121175622 / -34.9% 🟢 0.59 / 0.53 / +10.0%
tpcds_q36/datafusion:vortex-compact 54211702 / 70962680 / -23.6% 🟢 87904612 / 140880737 / -37.6% 🟢 0.62 / 0.50 / +22.4%
tpcds_q37/datafusion:vortex-compact 17615160 / 23146382 / -23.9% 🟢 47221507 / 74332622 / -36.5% 🟢 0.37 / 0.31 / +19.8%
tpcds_q38/datafusion:vortex-compact 35272962 / 46172920 / -23.6% 🟢 64030459 / 101225457 / -36.7% 🟢 0.55 / 0.46 / +20.8%
tpcds_q39/datafusion:vortex-compact 84072727 / 100089682 / -16.0% 🟢 113326396 / 156333290 / -27.5% 🟢 0.74 / 0.64 / +15.9%
tpcds_q40/datafusion:vortex-compact 27040866 / 36041886 / -25.0% 🟢 54996093 / 96163270 / -42.8% 🟢 0.49 / 0.37 / +31.2%
tpcds_q41/datafusion:vortex-compact 17997860 / 22210238 / -19.0% 🟢 41256957 / 56675339 / -27.2% 🟢 0.44 / 0.39 / +11.3%
tpcds_q42/datafusion:vortex-compact 13220040 / 17938902 / -26.3% 🟢 39487213 / 63083681 / -37.4% 🟢 0.33 / 0.28 / +17.7%
tpcds_q43/datafusion:vortex-compact 16706776 / 21728300 / -23.1% 🟢 44211309 / 70775445 / -37.5% 🟢 0.38 / 0.31 / +23.1%
tpcds_q44/datafusion:vortex-compact 34039194 / 46945254 / -27.5% 🟢 64795082 / 98081269 / -33.9% 🟢 0.53 / 0.48 / +9.8%
tpcds_q45/datafusion:vortex-compact 28654655 / 37627236 / -23.8% 🟢 57507511 / 87653864 / -34.4% 🟢 0.50 / 0.43 / +16.1%
tpcds_q46/datafusion:vortex-compact 30304184 / 42096621 / -28.0% 🟢 61950045 / 106741008 / -42.0% 🟢 0.49 / 0.39 / +24.0%
tpcds_q47/datafusion:vortex-compact 96206338 / 134498883 / -28.5% 🟢 130895430 / 210359253 / -37.8% 🟢 0.73 / 0.64 / +15.0%
tpcds_q48/datafusion:vortex-compact 53055434 / 93207850 / -43.1% 🟢 82227007 / 137945355 / -40.4% 🟢 0.65 / 0.68 / -4.5%
tpcds_q49/datafusion:vortex-compact 62741776 / 77064548 / -18.6% 🟢 97128802 / 133315987 / -27.1% 🟢 0.65 / 0.58 / +11.7%
tpcds_q50/datafusion:vortex-compact 36047008 / 48465614 / -25.6% 🟢 70192592 / 104396478 / -32.8% 🟢 0.51 / 0.46 / +10.6%
tpcds_q51/datafusion:vortex-compact 63871616 / 77109306 / -17.2% 🟢 98469041 / 135698499 / -27.4% 🟢 0.65 / 0.57 / +14.2%
tpcds_q52/datafusion:vortex-compact 13784168 / 18796274 / -26.7% 🟢 40327540 / 65476727 / -38.4% 🟢 0.34 / 0.29 / +19.1%
tpcds_q53/datafusion:vortex-compact 22816302 / 29233020 / -22.0% 🟢 52599665 / 81777294 / -35.7% 🟢 0.43 / 0.36 / +21.3%
tpcds_q54/datafusion:vortex-compact 37305513 / 47434492 / -21.4% 🟢 70108389 / 110098613 / -36.3% 🟢 0.53 / 0.43 / +23.5%
tpcds_q55/datafusion:vortex-compact 13639726 / 17795428 / -23.4% 🟢 40258073 / 65089440 / -38.1% 🟢 0.34 / 0.27 / +23.9%
tpcds_q56/datafusion:vortex-compact 28455309 / 38420383 / -25.9% 🟢 61214389 / 98485535 / -37.8% 🟢 0.46 / 0.39 / +19.2%
tpcds_q57/datafusion:vortex-compact 83972686 / 102378010 / -18.0% 🟢 117130014 / 173776329 / -32.6% 🟢 0.72 / 0.59 / +21.7%
tpcds_q58/datafusion:vortex-compact 48707454 / 64053870 / -24.0% 🟢 80908369 / 126442259 / -36.0% 🟢 0.60 / 0.51 / +18.8%
tpcds_q59/datafusion:vortex-compact 42635534 / 62093546 / -31.3% 🟢 69836669 / 116051231 / -39.8% 🟢 0.61 / 0.54 / +14.1%
tpcds_q60/datafusion:vortex-compact 29066570 / 35674642 / -18.5% 🟢 59468985 / 97195478 / -38.8% 🟢 0.49 / 0.37 / +33.2%
tpcds_q61/datafusion:vortex-compact 34894525 / 49556222 / -29.6% 🟢 63988225 / 109063556 / -41.3% 🟢 0.55 / 0.45 / +20.0%
tpcds_q62/datafusion:vortex-compact 18135459 / 25650920 / -29.3% 🟢 45024357 / 69685053 / -35.4% 🟢 0.40 / 0.37 / +9.4%
tpcds_q63/datafusion:vortex-compact 24026332 / 29672070 / -19.0% 🟢 53289948 / 82494725 / -35.4% 🟢 0.45 / 0.36 / +25.3%
tpcds_q64/datafusion:vortex-compact 445353357 / 516646908 / -13.8% 🟢 445258035 / 616975395 / -27.8% 🟢 1.00 / 0.84 / +19.4%
tpcds_q65/datafusion:vortex-compact 67781961 / 79775874 / -15.0% 🟢 97583734 / 138013345 / -29.3% 🟢 0.69 / 0.58 / +20.2%
tpcds_q66/datafusion:vortex-compact 62041392 / 74929967 / -17.2% 🟢 94714630 / 140139580 / -32.4% 🟢 0.66 / 0.53 / +22.5%
tpcds_q67/datafusion:vortex-compact 87821284 / 120394545 / -27.1% 🟢 129568281 / 199073423 / -34.9% 🟢 0.68 / 0.60 / +12.1%
tpcds_q68/datafusion:vortex-compact 31441178 / 43113000 / -27.1% 🟢 65343827 / 113091052 / -42.2% 🟢 0.48 / 0.38 / +26.2%
tpcds_q69/datafusion:vortex-compact 39725488 / 56321790 / -29.5% 🟢 71791083 / 107437942 / -33.2% 🟢 0.55 / 0.52 / +5.6%
tpcds_q70/datafusion:vortex-compact 82176741 / 116633186 / -29.5% 🟢 110632997 / 181274286 / -39.0% 🟢 0.74 / 0.64 / +15.4%
tpcds_q71/datafusion:vortex-compact 23292935 / 30623252 / -23.9% 🟢 51923016 / 88194396 / -41.1% 🟢 0.45 / 0.35 / +29.2%
tpcds_q72/datafusion:vortex-compact 1006381456 / 1474750360 / -31.8% 🟢 1006430262 / 1559768444 / -35.5% 🟢 1.00 / 0.95 / +5.8%
tpcds_q73/datafusion:vortex-compact 26805186 / 34007063 / -21.2% 🟢 59394187 / 91696864 / -35.2% 🟢 0.45 / 0.37 / +21.7%
tpcds_q74/datafusion:vortex-compact 56791764 / 80457416 / -29.4% 🟢 90328675 / 139640005 / -35.3% 🟢 0.63 / 0.58 / +9.1%
tpcds_q75/datafusion:vortex-compact 117150565 / 150364425 / -22.1% 🟢 155266213 / 229998685 / -32.5% 🟢 0.75 / 0.65 / +15.4%
tpcds_q76/datafusion:vortex-compact 37101648 / 45406590 / -18.3% 🟢 62415381 / 100281309 / -37.8% 🟢 0.59 / 0.45 / +31.3%
tpcds_q77/datafusion:vortex-compact 40614028 / 48902808 / -16.9% 🟢 73150299 / 113828807 / -35.7% 🟢 0.56 / 0.43 / +29.2%
tpcds_q78/datafusion:vortex-compact 83327448 / 120898254 / -31.1% 🟢 120086310 / 197753206 / -39.3% 🟢 0.69 / 0.61 / +13.5%
tpcds_q79/datafusion:vortex-compact 26737423 / 37445532 / -28.6% 🟢 56294480 / 90240884 / -37.6% 🟢 0.47 / 0.41 / +14.5%
tpcds_q80/datafusion:vortex-compact 78891188 / 104109736 / -24.2% 🟢 116868010 / 196317900 / -40.5% 🟢 0.68 / 0.53 / +27.3%
tpcds_q81/datafusion:vortex-compact 28732284 / 39152741 / -26.6% 🟢 56879192 / 89484817 / -36.4% 🟢 0.51 / 0.44 / +15.5%
tpcds_q82/datafusion:vortex-compact 17987562 / 24609220 / -26.9% 🟢 49322789 / 79270700 / -37.8% 🟢 0.36 / 0.31 / +17.5%
tpcds_q83/datafusion:vortex-compact 38019600 / 48750294 / -22.0% 🟢 65980427 / 102601283 / -35.7% 🟢 0.58 / 0.48 / +21.3%
tpcds_q84/datafusion:vortex-compact 13875132 / 17415576 / -20.3% 🟢 36982354 / 56883830 / -35.0% 🟢 0.38 / 0.31 / +22.5%
tpcds_q85/datafusion:vortex-compact 112791365 / 154116300 / -26.8% 🟢 146957038 / 223193379 / -34.2% 🟢 0.77 / 0.69 / +11.2%
tpcds_q86/datafusion:vortex-compact 15818834 / 21283784 / -25.7% 🟢 42450013 / 66020471 / -35.7% 🟢 0.37 / 0.32 / +15.6%
tpcds_q87/datafusion:vortex-compact 35369516 / 46684804 / -24.2% 🟢 63574769 / 102727560 / -38.1% 🟢 0.56 / 0.45 / +22.4%
tpcds_q88/datafusion:vortex-compact 58602172 / 81858232 / -28.4% 🟢 87849573 / 139443950 / -37.0% 🟢 0.67 / 0.59 / +13.6%
tpcds_q89/datafusion:vortex-compact 25963042 / 32121987 / -19.2% 🟢 55595424 / 86953530 / -36.1% 🟢 0.47 / 0.37 / +26.4%
tpcds_q90/datafusion:vortex-compact 13264092 / 16964402 / -21.8% 🟢 37808620 / 56685387 / -33.3% 🟢 0.35 / 0.30 / +17.2%
tpcds_q91/datafusion:vortex-compact 29898467 / 45348282 / -34.1% 🟢 56910334 / 91163447 / -37.6% 🟢 0.53 / 0.50 / +5.6%
tpcds_q92/datafusion:vortex-compact 15401994 / 20857004 / -26.2% 🟢 44171106 / 63286512 / -30.2% 🟢 0.35 / 0.33 / +5.8%
tpcds_q93/datafusion:vortex-compact 24113839 / 36082813 / -33.2% 🟢 50712629 / 88290818 / -42.6% 🟢 0.48 / 0.41 / +16.3%
tpcds_q94/datafusion:vortex-compact 21854577 / 29604542 / -26.2% 🟢 48602787 / 74332102 / -34.6% 🟢 0.45 / 0.40 / +12.9%
tpcds_q95/datafusion:vortex-compact 45286520 / 58818986 / -23.0% 🟢 74154766 / 120227361 / -38.3% 🟢 0.61 / 0.49 / +24.8%
tpcds_q96/datafusion:vortex-compact 12826642 / 17814422 / -28.0% 🟢 35952903 / 60685883 / -40.8% 🟢 0.36 / 0.29 / +21.5%
tpcds_q97/datafusion:vortex-compact 25384998 / 34591498 / -26.6% 🟢 54993836 / 90535898 / -39.3% 🟢 0.46 / 0.38 / +20.8%
tpcds_q98/datafusion:vortex-compact 21855186 / 29626467 / -26.2% 🟢 52628477 / 86110827 / -38.9% 🟢 0.42 / 0.34 / +20.7%
tpcds_q99/datafusion:vortex-compact 22649918 / 30088487 / -24.7% 🟢 49697513 / 81906815 / -39.3% 🟢 0.46 / 0.37 / +24.1%
datafusion / parquet / ns (0.782x ✅, 97↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpcds_q01/datafusion:parquet 27983626 / 36283692 / -22.9% 🟢 52567265 / 81234060 / -35.3% 🟢 0.53 / 0.45 / +19.2%
tpcds_q02/datafusion:parquet 39654548 / 52877966 / -25.0% 🟢 64928633 / 100451551 / -35.4% 🟢 0.61 / 0.53 / +16.0%
tpcds_q03/datafusion:parquet 16882131 / 22581489 / -25.2% 🟢 39118999 / 63535737 / -38.4% 🟢 0.43 / 0.36 / +21.4%
tpcds_q04/datafusion:parquet 223212239 / 280777708 / -20.5% 🟢 259796165 / 362304035 / -28.3% 🟢 0.86 / 0.77 / +10.9%
tpcds_q05/datafusion:parquet 57731704 / 71624132 / -19.4% 🟢 88507266 / 140691151 / -37.1% 🟢 0.65 / 0.51 / +28.1%
tpcds_q06/datafusion:parquet 29186316 / 38475778 / -24.1% 🟢 54978795 / 86422626 / -36.4% 🟢 0.53 / 0.45 / +19.2%
tpcds_q07/datafusion:parquet 39213435 / 57940085 / -32.3% 🟢 72526726 / 120774379 / -39.9% 🟢 0.54 / 0.48 / +12.7%
tpcds_q08/datafusion:parquet 34687079 / 42797697 / -19.0% 🟢 62114108 / 94597769 / -34.3% 🟢 0.56 / 0.45 / +23.4%
tpcds_q09/datafusion:parquet 27125940 / 32383394 / -16.2% 🟢 54262046 / 77246258 / -29.8% 🟢 0.50 / 0.42 / +19.2%
tpcds_q10/datafusion:parquet 43975708 / 55602183 / -20.9% 🟢 67790004 / 101912646 / -33.5% 🟢 0.65 / 0.55 / +18.9%
tpcds_q11/datafusion:parquet 102440051 / 144453106 / -29.1% 🟢 131255306 / 210640273 / -37.7% 🟢 0.78 / 0.69 / +13.8%
tpcds_q12/datafusion:parquet 22527052 / 30214202 / -25.4% 🟢 43798529 / 69908581 / -37.3% 🟢 0.51 / 0.43 / +19.0%
tpcds_q13/datafusion:parquet 41966834 / 58573133 / -28.4% 🟢 79043318 / 118560307 / -33.3% 🟢 0.53 / 0.49 / +7.5%
tpcds_q14/datafusion:parquet 137516956 / 176294868 / -22.0% 🟢 164805277 / 225681261 / -27.0% 🟢 0.83 / 0.78 / +6.8%
tpcds_q15/datafusion:parquet 25173748 / 30945960 / -18.7% 🟢 49407429 / 76992961 / -35.8% 🟢 0.51 / 0.40 / +26.8%
tpcds_q16/datafusion:parquet 30877612 / 38283082 / -19.3% 🟢 57289283 / 88265570 / -35.1% 🟢 0.54 / 0.43 / +24.3%
tpcds_q17/datafusion:parquet 61906358 / 77246826 / -19.9% 🟢 98374108 / 162847247 / -39.6% 🟢 0.63 / 0.47 / +32.7%
tpcds_q18/datafusion:parquet 48787006 / 65747741 / -25.8% 🟢 78145613 / 134021908 / -41.7% 🟢 0.62 / 0.49 / +27.3%
tpcds_q19/datafusion:parquet 29160274 / 37221003 / -21.7% 🟢 57088051 / 88647581 / -35.6% 🟢 0.51 / 0.42 / +21.7%
tpcds_q20/datafusion:parquet 23567832 / 34347300 / -31.4% 🟢 46579855 / 73308644 / -36.5% 🟢 0.51 / 0.47 / +8.0%
tpcds_q21/datafusion:parquet 21323317 / 25725864 / -17.1% 🟢 42475687 / 72040918 / -41.0% 🟢 0.50 / 0.36 / +40.6%
tpcds_q22/datafusion:parquet 90662571 / 120607942 / -24.8% 🟢 112279492 / 167107426 / -32.8% 🟢 0.81 / 0.72 / +11.9%
tpcds_q23/datafusion:parquet 117230628 / 158039522 / -25.8% 🟢 148721870 / 235205126 / -36.8% 🟢 0.79 / 0.67 / +17.3%
tpcds_q24/datafusion:parquet 78677442 / 101830010 / -22.7% 🟢 114065878 / 172738725 / -34.0% 🟢 0.69 / 0.59 / +17.0%
tpcds_q25/datafusion:parquet 64853648 / 83048156 / -21.9% 🟢 98756094 / 160565719 / -38.5% 🟢 0.66 / 0.52 / +27.0%
tpcds_q26/datafusion:parquet 36634418 / 46032848 / -20.4% 🟢 62407390 / 94875084 / -34.2% 🟢 0.59 / 0.49 / +21.0%
tpcds_q27/datafusion:parquet 76209759 / 110183226 / -30.8% 🟢 116885115 / 199269328 / -41.3% 🟢 0.65 / 0.55 / +17.9%
tpcds_q28/datafusion:parquet 26887546 / 36032952 / -25.4% 🟢 56184970 / 91318001 / -38.5% 🟢 0.48 / 0.39 / +21.3%
tpcds_q29/datafusion:parquet 63840698 / 81802098 / -22.0% 🟢 94820762 / 160806012 / -41.0% 🟢 0.67 / 0.51 / +32.4%
tpcds_q30/datafusion:parquet 45903896 / 55207702 / -16.9% 🟢 73672198 / 106720517 / -31.0% 🟢 0.62 / 0.52 / +20.4%
tpcds_q31/datafusion:parquet 76077030 / 94481100 / -19.5% 🟢 104753045 / 147563505 / -29.0% 🟢 0.73 / 0.64 / +13.4%
tpcds_q32/datafusion:parquet 22306315 / 26298434 / -15.2% 🟢 45433247 / 64774768 / -29.9% 🟢 0.49 / 0.41 / +20.9%
tpcds_q33/datafusion:parquet 35118666 / 43636254 / -19.5% 🟢 62822544 / 97184628 / -35.4% 🟢 0.56 / 0.45 / +24.5%
tpcds_q34/datafusion:parquet 22868876 / 30389886 / -24.7% 🟢 44533899 / 70247953 / -36.6% 🟢 0.51 / 0.43 / +18.7%
tpcds_q35/datafusion:parquet 53650082 / 71607714 / -25.1% 🟢 78896482 / 115920755 / -31.9% 🟢 0.68 / 0.62 / +10.1%
tpcds_q36/datafusion:parquet 49645550 / 67091992 / -26.0% 🟢 85003309 / 142192061 / -40.2% 🟢 0.58 / 0.47 / +23.8%
tpcds_q37/datafusion:parquet 21007262 / 24259836 / -13.4% 🟢 44154352 / 71755872 / -38.5% 🟢 0.48 / 0.34 / +40.7%
tpcds_q38/datafusion:parquet 40753768 / 51017540 / -20.1% 🟢 66572787 / 105734299 / -37.0% 🟢 0.61 / 0.48 / +26.9%
tpcds_q39/datafusion:parquet 77795452 / 85486639 / -9.0% 107140224 / 140727958 / -23.9% 🟢 0.73 / 0.61 / +19.5%
tpcds_q40/datafusion:parquet 27048580 / 32739752 / -17.4% 🟢 54758722 / 85011441 / -35.6% 🟢 0.49 / 0.39 / +28.3%
tpcds_q41/datafusion:parquet 19136798 / 23627780 / -19.0% 🟢 40460295 / 64877268 / -37.6% 🟢 0.47 / 0.36 / +29.9%
tpcds_q42/datafusion:parquet 15582896 / 20277914 / -23.2% 🟢 37105248 / 59110844 / -37.2% 🟢 0.42 / 0.34 / +22.4%
tpcds_q43/datafusion:parquet 18611728 / 25184346 / -26.1% 🟢 42525233 / 68600477 / -38.0% 🟢 0.44 / 0.37 / +19.2%
tpcds_q44/datafusion:parquet 31205344 / 39442540 / -20.9% 🟢 62106725 / 94832200 / -34.5% 🟢 0.50 / 0.42 / +20.8%
tpcds_q45/datafusion:parquet 31745660 / 39900652 / -20.4% 🟢 55982645 / 98802099 / -43.3% 🟢 0.57 / 0.40 / +40.4%
tpcds_q46/datafusion:parquet 29274686 / 39310170 / -25.5% 🟢 61373344 / 98593936 / -37.8% 🟢 0.48 / 0.40 / +19.6%
tpcds_q47/datafusion:parquet 101603156 / 142277840 / -28.6% 🟢 133364709 / 211747821 / -37.0% 🟢 0.76 / 0.67 / +13.4%
tpcds_q48/datafusion:parquet 37423456 / 53554153 / -30.1% 🟢 66952838 / 107030588 / -37.4% 🟢 0.56 / 0.50 / +11.7%
tpcds_q49/datafusion:parquet 61780142 / 76721002 / -19.5% 🟢 95733662 / 141289799 / -32.2% 🟢 0.65 / 0.54 / +18.8%
tpcds_q50/datafusion:parquet 43555989 / 56933463 / -23.5% 🟢 70184306 / 116219502 / -39.6% 🟢 0.62 / 0.49 / +26.7%
tpcds_q51/datafusion:parquet 70664791 / 82729314 / -14.6% 🟢 98196991 / 142845366 / -31.3% 🟢 0.72 / 0.58 / +24.3%
tpcds_q52/datafusion:parquet 16325296 / 20589010 / -20.7% 🟢 38162744 / 57689357 / -33.8% 🟢 0.43 / 0.36 / +19.9%
tpcds_q53/datafusion:parquet 20784572 / 26062956 / -20.3% 🟢 47357912 / 73464191 / -35.5% 🟢 0.44 / 0.35 / +23.7%
tpcds_q54/datafusion:parquet 46153882 / 53932054 / -14.4% 🟢 75047223 / 102302283 / -26.6% 🟢 0.61 / 0.53 / +16.7%
tpcds_q55/datafusion:parquet 15959270 / 21418592 / -25.5% 🟢 37169127 / 59302756 / -37.3% 🟢 0.43 / 0.36 / +18.9%
tpcds_q56/datafusion:parquet 38855106 / 44705716 / -13.1% 🟢 64584190 / 100978120 / -36.0% 🟢 0.60 / 0.44 / +35.9%
tpcds_q57/datafusion:parquet 101164582 / 118766058 / -14.8% 🟢 126162545 / 182338414 / -30.8% 🟢 0.80 / 0.65 / +23.1%
tpcds_q58/datafusion:parquet 58218747 / 73181658 / -20.4% 🟢 84275151 / 134192417 / -37.2% 🟢 0.69 / 0.55 / +26.7%
tpcds_q59/datafusion:parquet 46469406 / 70431544 / -34.0% 🟢 71259167 / 123796762 / -42.4% 🟢 0.65 / 0.57 / +14.6%
tpcds_q60/datafusion:parquet 38896836 / 44607015 / -12.8% 🟢 60786667 / 98218333 / -38.1% 🟢 0.64 / 0.45 / +40.9%
tpcds_q61/datafusion:parquet 40564940 / 52348038 / -22.5% 🟢 70343026 / 113930720 / -38.3% 🟢 0.58 / 0.46 / +25.5%
tpcds_q62/datafusion:parquet 24733722 / 28315156 / -12.6% 🟢 45685612 / 81804126 / -44.2% 🟢 0.54 / 0.35 / +56.4%
tpcds_q63/datafusion:parquet 20202460 / 26450762 / -23.6% 🟢 45906754 / 72440757 / -36.6% 🟢 0.44 / 0.37 / +20.5%
tpcds_q64/datafusion:parquet 213184960 / 281788878 / -24.3% 🟢 252278192 / 358645582 / -29.7% 🟢 0.85 / 0.79 / +7.6%
tpcds_q65/datafusion:parquet 31547586 / 43417883 / -27.3% 🟢 60047830 / 100451153 / -40.2% 🟢 0.53 / 0.43 / +21.6%
tpcds_q66/datafusion:parquet 69641250 / 82983548 / -16.1% 🟢 92215031 / 128231487 / -28.1% 🟢 0.76 / 0.65 / +16.7%
tpcds_q67/datafusion:parquet 94274216 / 136869266 / -31.1% 🟢 127872528 / 215247417 / -40.6% 🟢 0.74 / 0.64 / +15.9%
tpcds_q68/datafusion:parquet 28829789 / 37565831 / -23.3% 🟢 67775544 / 101982411 / -33.5% 🟢 0.43 / 0.37 / +15.5%
tpcds_q69/datafusion:parquet 45476582 / 58967198 / -22.9% 🟢 68960570 / 98865683 / -30.2% 🟢 0.66 / 0.60 / +10.6%
tpcds_q70/datafusion:parquet 34108711 / 48888860 / -30.2% 🟢 63770472 / 103840029 / -38.6% 🟢 0.53 / 0.47 / +13.6%
tpcds_q71/datafusion:parquet 26250716 / 34176094 / -23.2% 🟢 53697488 / 81939803 / -34.5% 🟢 0.49 / 0.42 / +17.2%
tpcds_q72/datafusion:parquet 259021924 / 392164832 / -34.0% 🟢 282300985 / 521481982 / -45.9% 🟢 0.92 / 0.75 / +22.0%
tpcds_q73/datafusion:parquet 22455400 / 30888395 / -27.3% 🟢 45013198 / 78361860 / -42.6% 🟢 0.50 / 0.39 / +26.6%
tpcds_q74/datafusion:parquet 68914454 / 91160091 / -24.4% 🟢 92988929 / 146911802 / -36.7% 🟢 0.74 / 0.62 / +19.4%
tpcds_q75/datafusion:parquet 116989998 / 148211452 / -21.1% 🟢 154835878 / 233617221 / -33.7% 🟢 0.76 / 0.63 / +19.1%
tpcds_q76/datafusion:parquet 39047260 / 47707535 / -18.2% 🟢 64618690 / 110712841 / -41.6% 🟢 0.60 / 0.43 / +40.2%
tpcds_q77/datafusion:parquet 48927013 / 57484674 / -14.9% 🟢 76415577 / 120750544 / -36.7% 🟢 0.64 / 0.48 / +34.5%
tpcds_q78/datafusion:parquet 85085488 / 117182664 / -27.4% 🟢 132770381 / 213467559 / -37.8% 🟢 0.64 / 0.55 / +16.7%
tpcds_q79/datafusion:parquet 25777352 / 34575099 / -25.4% 🟢 53461196 / 89234208 / -40.1% 🟢 0.48 / 0.39 / +24.4%
tpcds_q80/datafusion:parquet 73560108 / 96163958 / -23.5% 🟢 113959143 / 197855350 / -42.4% 🟢 0.65 / 0.49 / +32.8%
tpcds_q81/datafusion:parquet 42066702 / 51436269 / -18.2% 🟢 66168261 / 98387095 / -32.7% 🟢 0.64 / 0.52 / +21.6%
tpcds_q82/datafusion:parquet 21843695 / 27937386 / -21.8% 🟢 43537528 / 74375263 / -41.5% 🟢 0.50 / 0.38 / +33.6%
tpcds_q83/datafusion:parquet 50140142 / 61363235 / -18.3% 🟢 77123416 / 109377944 / -29.5% 🟢 0.65 / 0.56 / +15.9%
tpcds_q84/datafusion:parquet 28186215 / 34289137 / -17.8% 🟢 50527175 / 76296688 / -33.8% 🟢 0.56 / 0.45 / +24.1%
tpcds_q85/datafusion:parquet 68109949 / 90019107 / -24.3% 🟢 103832642 / 166745754 / -37.7% 🟢 0.66 / 0.54 / +21.5%
tpcds_q86/datafusion:parquet 18261700 / 22827416 / -20.0% 🟢 38213665 / 61768504 / -38.1% 🟢 0.48 / 0.37 / +29.3%
tpcds_q87/datafusion:parquet 39451082 / 48635282 / -18.9% 🟢 63960014 / 102330814 / -37.5% 🟢 0.62 / 0.48 / +29.8%
tpcds_q88/datafusion:parquet 49058180 / 56562930 / -13.3% 🟢 71371869 / 119406901 / -40.2% 🟢 0.69 / 0.47 / +45.1%
tpcds_q89/datafusion:parquet 23964950 / 29788615 / -19.5% 🟢 49688987 / 80463630 / -38.2% 🟢 0.48 / 0.37 / +30.3%
tpcds_q90/datafusion:parquet 17797846 / 21309314 / -16.5% 🟢 38015733 / 58415079 / -34.9% 🟢 0.47 / 0.36 / +28.3%
tpcds_q91/datafusion:parquet 35018775 / 43979552 / -20.4% 🟢 57826958 / 83756452 / -31.0% 🟢 0.61 / 0.53 / +15.3%
tpcds_q92/datafusion:parquet 22626844 / 26182258 / -13.6% 🟢 44964805 / 69886319 / -35.7% 🟢 0.50 / 0.37 / +34.3%
tpcds_q93/datafusion:parquet 24951558 / 34234256 / -27.1% 🟢 52149701 / 92365755 / -43.5% 🟢 0.48 / 0.37 / +29.1%
tpcds_q94/datafusion:parquet 29251946 / 35285800 / -17.1% 🟢 52768279 / 77311002 / -31.7% 🟢 0.55 / 0.46 / +21.5%
tpcds_q95/datafusion:parquet 49765267 / 61284348 / -18.8% 🟢 75251646 / 117403935 / -35.9% 🟢 0.66 / 0.52 / +26.7%
tpcds_q96/datafusion:parquet 14655366 / 18915406 / -22.5% 🟢 35031691 / 54079231 / -35.2% 🟢 0.42 / 0.35 / +19.6%
tpcds_q97/datafusion:parquet 32122530 / 40342503 / -20.4% 🟢 56512271 / 88439786 / -36.1% 🟢 0.57 / 0.46 / +24.6%
tpcds_q98/datafusion:parquet 23793542 / 30659179 / -22.4% 🟢 47957790 / 77855186 / -38.4% 🟢 0.50 / 0.39 / +26.0%
tpcds_q99/datafusion:parquet 30675102 / 33649148 / -8.8% 51421106 / 76168384 / -32.5% 🟢 0.60 / 0.44 / +35.0%
duckdb / vortex-file-compressed / ns (0.708x ✅, 99↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpcds_q01/duckdb:vortex-file-compressed 19472926 / 23709576 / -17.9% 🟢 29961695 / 45473646 / -34.1% 🟢 0.65 / 0.52 / +24.7%
tpcds_q02/duckdb:vortex-file-compressed 17267025 / 28665785 / -39.8% 🟢 25666751 / 47665966 / -46.2% 🟢 0.67 / 0.60 / +11.9%
tpcds_q03/duckdb:vortex-file-compressed 13477739 / 19802494 / -31.9% 🟢 27031293 / 41494092 / -34.9% 🟢 0.50 / 0.48 / +4.5%
tpcds_q04/duckdb:vortex-file-compressed 65744874 / 90837958 / -27.6% 🟢 82747740 / 121843565 / -32.1% 🟢 0.79 / 0.75 / +6.6%
tpcds_q05/duckdb:vortex-file-compressed 24373366 / 34150462 / -28.6% 🟢 34929608 / 55149800 / -36.7% 🟢 0.70 / 0.62 / +12.7%
tpcds_q06/duckdb:vortex-file-compressed 33374034 / 43504298 / -23.3% 🟢 51568407 / 70228543 / -26.6% 🟢 0.65 / 0.62 / +4.5%
tpcds_q07/duckdb:vortex-file-compressed 18726760 / 29540912 / -36.6% 🟢 36143819 / 61914393 / -41.6% 🟢 0.52 / 0.48 / +8.6%
tpcds_q08/duckdb:vortex-file-compressed 28138526 / 37249331 / -24.5% 🟢 42120748 / 57310150 / -26.5% 🟢 0.67 / 0.65 / +2.8%
tpcds_q09/duckdb:vortex-file-compressed 13906127 / 17052058 / -18.4% 🟢 23987746 / 52798344 / -54.6% 🟢 0.58 / 0.32 / +79.5%
tpcds_q10/duckdb:vortex-file-compressed 35623952 / 49877650 / -28.6% 🟢 51904428 / 75553489 / -31.3% 🟢 0.69 / 0.66 / +4.0%
tpcds_q11/duckdb:vortex-file-compressed 50685284 / 72748082 / -30.3% 🟢 66470570 / 108433807 / -38.7% 🟢 0.76 / 0.67 / +13.7%
tpcds_q12/duckdb:vortex-file-compressed 13202156 / 17154959 / -23.0% 🟢 23235026 / 33087230 / -29.8% 🟢 0.57 / 0.52 / +9.6%
tpcds_q13/duckdb:vortex-file-compressed 25017110 / 36737458 / -31.9% 🟢 45255898 / 70215132 / -35.5% 🟢 0.55 / 0.52 / +5.7%
tpcds_q14/duckdb:vortex-file-compressed 74213336 / 104653262 / -29.1% 🟢 84666179 / 152975612 / -44.7% 🟢 0.88 / 0.68 / +28.1%
tpcds_q15/duckdb:vortex-file-compressed 28040172 / 33338778 / -15.9% 🟢 35568250 / 47169573 / -24.6% 🟢 0.79 / 0.71 / +11.5%
tpcds_q16/duckdb:vortex-file-compressed 22909522 / 31765316 / -27.9% 🟢 34849717 / 53185679 / -34.5% 🟢 0.66 / 0.60 / +10.1%
tpcds_q17/duckdb:vortex-file-compressed 38597080 / 50770618 / -24.0% 🟢 57886964 / 87769112 / -34.0% 🟢 0.67 / 0.58 / +15.3%
tpcds_q18/duckdb:vortex-file-compressed 35342550 / 47072822 / -24.9% 🟢 49812011 / 71505577 / -30.3% 🟢 0.71 / 0.66 / +7.8%
tpcds_q19/duckdb:vortex-file-compressed 30527460 / 39068493 / -21.9% 🟢 45928024 / 64696633 / -29.0% 🟢 0.66 / 0.60 / +10.1%
tpcds_q20/duckdb:vortex-file-compressed 14235448 / 18026558 / -21.0% 🟢 21912692 / 37481682 / -41.5% 🟢 0.65 / 0.48 / +35.1%
tpcds_q21/duckdb:vortex-file-compressed 12406771 / 22175640 / -44.1% 🟢 25669057 / 48237237 / -46.8% 🟢 0.48 / 0.46 / +5.1%
tpcds_q22/duckdb:vortex-file-compressed 39338988 / 67862966 / -42.0% 🟢 56318177 / 104543256 / -46.1% 🟢 0.70 / 0.65 / +7.6%
tpcds_q23/duckdb:vortex-file-compressed 47565578 / 69605258 / -31.7% 🟢 64963273 / 112087274 / -42.0% 🟢 0.73 / 0.62 / +17.9%
tpcds_q24/duckdb:vortex-file-compressed 36395249 / 51179688 / -28.9% 🟢 50854721 / 78858454 / -35.5% 🟢 0.72 / 0.65 / +10.3%
tpcds_q25/duckdb:vortex-file-compressed 32409043 / 43156450 / -24.9% 🟢 50290233 / 69837354 / -28.0% 🟢 0.64 / 0.62 / +4.3%
tpcds_q26/duckdb:vortex-file-compressed 17952860 / 22155674 / -19.0% 🟢 26448112 / 40895289 / -35.3% 🟢 0.68 / 0.54 / +25.3%
tpcds_q27/duckdb:vortex-file-compressed 20006072 / 29939928 / -33.2% 🟢 34294404 / 60964665 / -43.7% 🟢 0.58 / 0.49 / +18.8%
tpcds_q28/duckdb:vortex-file-compressed 10085534 / 12731146 / -20.8% 🟢 17088926 / 24384024 / -29.9% 🟢 0.59 / 0.52 / +13.0%
tpcds_q29/duckdb:vortex-file-compressed 36861260 / 47215808 / -21.9% 🟢 57615011 / 83507089 / -31.0% 🟢 0.64 / 0.57 / +13.2%
tpcds_q30/duckdb:vortex-file-compressed 24490622 / 31973519 / -23.4% 🟢 36367819 / 53518789 / -32.0% 🟢 0.67 / 0.60 / +12.7%
tpcds_q31/duckdb:vortex-file-compressed 24691868 / 32757864 / -24.6% 🟢 36015117 / 57352261 / -37.2% 🟢 0.69 / 0.57 / +20.0%
tpcds_q32/duckdb:vortex-file-compressed 13353948 / 18437780 / -27.6% 🟢 22536653 / 32007088 / -29.6% 🟢 0.59 / 0.58 / +2.9%
tpcds_q33/duckdb:vortex-file-compressed 20591999 / 28888605 / -28.7% 🟢 36688993 / 59318293 / -38.1% 🟢 0.56 / 0.49 / +15.2%
tpcds_q34/duckdb:vortex-file-compressed 17041075 / 24441713 / -30.3% 🟢 29548060 / 44961369 / -34.3% 🟢 0.58 / 0.54 / +6.1%
tpcds_q35/duckdb:vortex-file-compressed 60141526 / 76518793 / -21.4% 🟢 75322932 / 114588619 / -34.3% 🟢 0.80 / 0.67 / +19.6%
tpcds_q36/duckdb:vortex-file-compressed 18117832 / 27754997 / -34.7% 🟢 34568002 / 57843516 / -40.2% 🟢 0.52 / 0.48 / +9.2%
tpcds_q37/duckdb:vortex-file-compressed 16465222 / 28837185 / -42.9% 🟢 24867923 / 54170091 / -54.1% 🟢 0.66 / 0.53 / +24.4%
tpcds_q38/duckdb:vortex-file-compressed 27940565 / 38474907 / -27.4% 🟢 35346468 / 66927955 / -47.2% 🟢 0.79 / 0.57 / +37.5%
tpcds_q39/duckdb:vortex-file-compressed 23460574 / 32889260 / -28.7% 🟢 37247467 / 63749803 / -41.6% 🟢 0.63 / 0.52 / +22.1%
tpcds_q40/duckdb:vortex-file-compressed 16995012 / 22442342 / -24.3% 🟢 24029101 / 41130330 / -41.6% 🟢 0.71 / 0.55 / +29.6%
tpcds_q41/duckdb:vortex-file-compressed 8429309 / 9562816 / -11.9% 🟢 13205353 / 19729302 / -33.1% 🟢 0.64 / 0.48 / +31.7%
tpcds_q42/duckdb:vortex-file-compressed 10007654 / 16184114 / -38.2% 🟢 24579623 / 39858918 / -38.3% 🟢 0.41 / 0.41 / +0.3%
tpcds_q43/duckdb:vortex-file-compressed 11762900 / 19284900 / -39.0% 🟢 23545008 / 39399116 / -40.2% 🟢 0.50 / 0.49 / +2.1%
tpcds_q44/duckdb:vortex-file-compressed 17758341 / 24363630 / -27.1% 🟢 29983679 / 62613093 / -52.1% 🟢 0.59 / 0.39 / +52.2%
tpcds_q45/duckdb:vortex-file-compressed 30013678 / 35567907 / -15.6% 🟢 38152404 / 49668176 / -23.2% 🟢 0.79 / 0.72 / +9.9%
tpcds_q46/duckdb:vortex-file-compressed 20727000 / 33724319 / -38.5% 🟢 36614781 / 59553006 / -38.5% 🟢 0.57 / 0.57 / -0.0%
tpcds_q47/duckdb:vortex-file-compressed 37935453 / 53109812 / -28.6% 🟢 52487606 / 85386712 / -38.5% 🟢 0.72 / 0.62 / +16.2%
tpcds_q48/duckdb:vortex-file-compressed 23574772 / 34502634 / -31.7% 🟢 39968953 / 68689799 / -41.8% 🟢 0.59 / 0.50 / +17.4%
tpcds_q49/duckdb:vortex-file-compressed 23585645 / 30748593 / -23.3% 🟢 40322099 / 71822576 / -43.9% 🟢 0.58 / 0.43 / +36.6%
tpcds_q50/duckdb:vortex-file-compressed 20883901 / 29127540 / -28.3% 🟢 38142934 / 57940859 / -34.2% 🟢 0.55 / 0.50 / +8.9%
tpcds_q51/duckdb:vortex-file-compressed 68002340 / 131098275 / -48.1% 🟢 83765973 / 122077931 / -31.4% 🟢 0.81 / 1.07 / -24.4%
tpcds_q52/duckdb:vortex-file-compressed 10349542 / 17123469 / -39.6% 🟢 24596050 / 39487383 / -37.7% 🟢 0.42 / 0.43 / -3.0%
tpcds_q53/duckdb:vortex-file-compressed 17995078 / 25109466 / -28.3% 🟢 32803637 / 48675878 / -32.6% 🟢 0.55 / 0.52 / +6.3%
tpcds_q54/duckdb:vortex-file-compressed 25308117 / 37070228 / -31.7% 🟢 39103093 / 59797236 / -34.6% 🟢 0.65 / 0.62 / +4.4%
tpcds_q55/duckdb:vortex-file-compressed 10268266 / 18073067 / -43.2% 🟢 25949542 / 42502079 / -38.9% 🟢 0.40 / 0.43 / -6.9%
tpcds_q56/duckdb:vortex-file-compressed 20939508 / 31368422 / -33.2% 🟢 38048784 / 61712454 / -38.3% 🟢 0.55 / 0.51 / +8.3%
tpcds_q57/duckdb:vortex-file-compressed 31436442 / 40160761 / -21.7% 🟢 41297604 / 61887419 / -33.3% 🟢 0.76 / 0.65 / +17.3%
tpcds_q58/duckdb:vortex-file-compressed 22495616 / 32213514 / -30.2% 🟢 35853042 / 58875383 / -39.1% 🟢 0.63 / 0.55 / +14.7%
tpcds_q59/duckdb:vortex-file-compressed 21720344 / 38362720 / -43.4% 🟢 32669609 / 56974417 / -42.7% 🟢 0.66 / 0.67 / -1.3%
tpcds_q60/duckdb:vortex-file-compressed 21941830 / 30470667 / -28.0% 🟢 38769716 / 62789942 / -38.3% 🟢 0.57 / 0.49 / +16.6%
tpcds_q61/duckdb:vortex-file-compressed 25591557 / 35910150 / -28.7% 🟢 41701253 / 80373312 / -48.1% 🟢 0.61 / 0.45 / +37.4%
tpcds_q62/duckdb:vortex-file-compressed 12690359 / 18329504 / -30.8% 🟢 21554514 / 34088346 / -36.8% 🟢 0.59 / 0.54 / +9.5%
tpcds_q63/duckdb:vortex-file-compressed 16795184 / 21881221 / -23.2% 🟢 31079601 / 51063425 / -39.1% 🟢 0.54 / 0.43 / +26.1%
tpcds_q64/duckdb:vortex-file-compressed 78011250 / 107991262 / -27.8% 🟢 96457878 / 150616713 / -36.0% 🟢 0.81 / 0.72 / +12.8%
tpcds_q65/duckdb:vortex-file-compressed 16670820 / 23388496 / -28.7% 🟢 27899653 / 45472205 / -38.6% 🟢 0.60 / 0.51 / +16.2%
tpcds_q66/duckdb:vortex-file-compressed 27516123 / 37530986 / -26.7% 🟢 37885028 / 68646491 / -44.8% 🟢 0.73 / 0.55 / +32.8%
tpcds_q67/duckdb:vortex-file-compressed 85223947 / 124453922 / -31.5% 🟢 108619847 / 164865234 / -34.1% 🟢 0.78 / 0.75 / +3.9%
tpcds_q68/duckdb:vortex-file-compressed 20554020 / 29679482 / -30.7% 🟢 37045218 / 62232455 / -40.5% 🟢 0.55 / 0.48 / +16.3%
tpcds_q69/duckdb:vortex-file-compressed 37270467 / 52722828 / -29.3% 🟢 54035232 / 84485585 / -36.0% 🟢 0.69 / 0.62 / +10.5%
tpcds_q70/duckdb:vortex-file-compressed 22620539 / 32817190 / -31.1% 🟢 36154142 / 67063160 / -46.1% 🟢 0.63 / 0.49 / +27.9%
tpcds_q71/duckdb:vortex-file-compressed 16749506 / 24158307 / -30.7% 🟢 31554646 / 56582306 / -44.2% 🟢 0.53 / 0.43 / +24.3%
tpcds_q72/duckdb:vortex-file-compressed 100697577 / 143065148 / -29.6% 🟢 102142567 / 191621687 / -46.7% 🟢 0.99 / 0.75 / +32.0%
tpcds_q73/duckdb:vortex-file-compressed 16616694 / 24204096 / -31.3% 🟢 27336624 / 44665728 / -38.8% 🟢 0.61 / 0.54 / +12.2%
tpcds_q74/duckdb:vortex-file-compressed 35158966 / 49336790 / -28.7% 🟢 48050810 / 74592836 / -35.6% 🟢 0.73 / 0.66 / +10.6%
tpcds_q75/duckdb:vortex-file-compressed 35825868 / 53109449 / -32.5% 🟢 49408583 / 87002623 / -43.2% 🟢 0.73 / 0.61 / +18.8%
tpcds_q76/duckdb:vortex-file-compressed 16699152 / 23510192 / -29.0% 🟢 29138722 / 56647436 / -48.6% 🟢 0.57 / 0.42 / +38.1%
tpcds_q77/duckdb:vortex-file-compressed 19610468 / 27532720 / -28.8% 🟢 32495885 / 55239078 / -41.2% 🟢 0.60 / 0.50 / +21.1%
tpcds_q78/duckdb:vortex-file-compressed 48234908 / 70691455 / -31.8% 🟢 59334774 / 101454157 / -41.5% 🟢 0.81 / 0.70 / +16.7%
tpcds_q79/duckdb:vortex-file-compressed 17538568 / 28078773 / -37.5% 🟢 32634209 / 57576807 / -43.3% 🟢 0.54 / 0.49 / +10.2%
tpcds_q80/duckdb:vortex-file-compressed 34121554 / 50679260 / -32.7% 🟢 51878463 / 86963853 / -40.3% 🟢 0.66 / 0.58 / +12.9%
tpcds_q81/duckdb:vortex-file-compressed 28359998 / 39268564 / -27.8% 🟢 40362772 / 58110497 / -30.5% 🟢 0.70 / 0.68 / +4.0%
tpcds_q82/duckdb:vortex-file-compressed 38541598 / 53163056 / -27.5% 🟢 47682252 / 74990170 / -36.4% 🟢 0.81 / 0.71 / +14.0%
tpcds_q83/duckdb:vortex-file-compressed 26505390 / 31351054 / -15.5% 🟢 32709870 / 47164295 / -30.6% 🟢 0.81 / 0.66 / +21.9%
tpcds_q84/duckdb:vortex-file-compressed 15817408 / 25114798 / -37.0% 🟢 23525394 / 40620168 / -42.1% 🟢 0.67 / 0.62 / +8.7%
tpcds_q85/duckdb:vortex-file-compressed 40065356 / 50113500 / -20.1% 🟢 54054551 / 75328630 / -28.2% 🟢 0.74 / 0.67 / +11.4%
tpcds_q86/duckdb:vortex-file-compressed 13950108 / 20385974 / -31.6% 🟢 23470615 / 37666973 / -37.7% 🟢 0.59 / 0.54 / +9.8%
tpcds_q87/duckdb:vortex-file-compressed 29829466 / 40982276 / -27.2% 🟢 38370774 / 66641901 / -42.4% 🟢 0.78 / 0.61 / +26.4%
tpcds_q88/duckdb:vortex-file-compressed 50075780 / 61275280 / -18.3% 🟢 60727624 / 122101642 / -50.3% 🟢 0.82 / 0.50 / +64.3%
tpcds_q89/duckdb:vortex-file-compressed 16905286 / 25858351 / -34.6% 🟢 31062665 / 50324465 / -38.3% 🟢 0.54 / 0.51 / +5.9%
tpcds_q90/duckdb:vortex-file-compressed 10115584 / 14908972 / -32.2% 🟢 16092447 / 29418461 / -45.3% 🟢 0.63 / 0.51 / +24.0%
tpcds_q91/duckdb:vortex-file-compressed 20381002 / 27933116 / -27.0% 🟢 30044419 / 45287930 / -33.7% 🟢 0.68 / 0.62 / +10.0%
tpcds_q92/duckdb:vortex-file-compressed 17515136 / 21034046 / -16.7% 🟢 28748385 / 43806516 / -34.4% 🟢 0.61 / 0.48 / +26.9%
tpcds_q93/duckdb:vortex-file-compressed 20234666 / 29016604 / -30.3% 🟢 30596648 / 48436699 / -36.8% 🟢 0.66 / 0.60 / +10.4%
tpcds_q94/duckdb:vortex-file-compressed 21214935 / 30601982 / -30.7% 🟢 32525253 / 50242083 / -35.3% 🟢 0.65 / 0.61 / +7.1%
tpcds_q95/duckdb:vortex-file-compressed 89179094 / 141438531 / -36.9% 🟢 97923296 / 159075523 / -38.4% 🟢 0.91 / 0.89 / +2.4%
tpcds_q96/duckdb:vortex-file-compressed 10317925 / 14715449 / -29.9% 🟢 21523554 / 39853986 / -46.0% 🟢 0.48 / 0.37 / +29.8%
tpcds_q97/duckdb:vortex-file-compressed 29075311 / 38441851 / -24.4% 🟢 38926182 / 67715885 / -42.5% 🟢 0.75 / 0.57 / +31.6%
tpcds_q98/duckdb:vortex-file-compressed 16705556 / 22571805 / -26.0% 🟢 29488086 / 51643326 / -42.9% 🟢 0.57 / 0.44 / +29.6%
tpcds_q99/duckdb:vortex-file-compressed 17311768 / 24589096 / -29.6% 🟢 25850568 / 40944674 / -36.9% 🟢 0.67 / 0.60 / +11.5%
duckdb / vortex-compact / ns (0.668x ✅, 98↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpcds_q01/duckdb:vortex-compact 20608123 / 25635586 / -19.6% 🟢 31100072 / 46023851 / -32.4% 🟢 0.66 / 0.56 / +19.0%
tpcds_q02/duckdb:vortex-compact 18403875 / 30297914 / -39.3% 🟢 26769957 / 48056366 / -44.3% 🟢 0.69 / 0.63 / +9.0%
tpcds_q03/duckdb:vortex-compact 25934924 / 44949454 / -42.3% 🟢 37582456 / 65865765 / -42.9% 🟢 0.69 / 0.68 / +1.1%
tpcds_q04/duckdb:vortex-compact 69399637 / 97910750 / -29.1% 🟢 82453953 / 133302144 / -38.1% 🟢 0.84 / 0.73 / +14.6%
tpcds_q05/duckdb:vortex-compact 28232488 / 42218186 / -33.1% 🟢 38744250 / 68436374 / -43.4% 🟢 0.73 / 0.62 / +18.1%
tpcds_q06/duckdb:vortex-compact 39670034 / 57887740 / -31.5% 🟢 55827594 / 84670647 / -34.1% 🟢 0.71 / 0.68 / +3.9%
tpcds_q07/duckdb:vortex-compact 33177312 / 5847766 / -43.3% 🟢 47113653 / 95741346 / -50.8% 🟢 0.70 / 0.61 / +15.3%
tpcds_q08/duckdb:vortex-compact 61015008 / 99443308 / -38.6% 🟢 72071668 / 115254512 / -37.5% 🟢 0.85 / 0.86 / -1.9%
tpcds_q09/duckdb:vortex-compact 17156829 / 23347440 / -26.5% 🟢 27573908 / 64017217 / -56.9% 🟢 0.62 / 0.36 / +70.6%
tpcds_q10/duckdb:vortex-compact 54644164 / 86662694 / -36.9% 🟢 68617002 / 113412137 / -39.5% 🟢 0.80 / 0.76 / +4.2%
tpcds_q11/duckdb:vortex-compact 55623876 / 84854852 / -34.4% 🟢 67225917 / 102137514 / -34.2% 🟢 0.83 / 0.83 / -0.4%
tpcds_q12/duckdb:vortex-compact 16490459 / 22052295 / -25.2% 🟢 25026186 / 36241345 / -30.9% 🟢 0.66 / 0.61 / +8.3%
tpcds_q13/duckdb:vortex-compact 43560488 / 77008837 / -43.4% 🟢 59893082 / 107577300 / -44.3% 🟢 0.73 / 0.72 / +1.6%
tpcds_q14/duckdb:vortex-compact 84056698 / 125942272 / -33.3% 🟢 99797154 / 175591438 / -43.2% 🟢 0.84 / 0.72 / +17.4%
tpcds_q15/duckdb:vortex-compact 32844689 / 39919143 / -17.7% 🟢 38201411 / 54579152 / -30.0% 🟢 0.86 / 0.73 / +17.6%
tpcds_q16/duckdb:vortex-compact 30352659 / 41008250 / -26.0% 🟢 41665826 / 61692055 / -32.5% 🟢 0.73 / 0.66 / +9.6%
tpcds_q17/duckdb:vortex-compact 55237432 / 81375946 / -32.1% 🟢 71475538 / 108223257 / -34.0% 🟢 0.77 / 0.75 / +2.8%
tpcds_q18/duckdb:vortex-compact 43363632 / 60380968 / -28.2% 🟢 57141451 / 82898982 / -31.1% 🟢 0.76 / 0.73 / +4.2%
tpcds_q19/duckdb:vortex-compact 44036980 / 63277306 / -30.4% 🟢 53481838 / 86971449 / -38.5% 🟢 0.82 / 0.73 / +13.2%
tpcds_q20/duckdb:vortex-compact 17788488 / 23120612 / -23.1% 🟢 25078364 / 42034280 / -40.3% 🟢 0.71 / 0.55 / +29.0%
tpcds_q21/duckdb:vortex-compact 13087428 / 21644128 / -39.5% 🟢 24650430 / 62608469 / -60.6% 🟢 0.53 / 0.35 / +53.6%
tpcds_q22/duckdb:vortex-compact 40776978 / 67016816 / -39.2% 🟢 54436724 / 114256671 / -52.4% 🟢 0.75 / 0.59 / +27.7%
tpcds_q23/duckdb:vortex-compact 59454419 / 88557290 / -32.9% 🟢 71244844 / 132746289 / -46.3% 🟢 0.83 / 0.67 / +25.1%
tpcds_q24/duckdb:vortex-compact 49680282 / 76759073 / -35.3% 🟢 62385889 / 100147767 / -37.7% 🟢 0.80 / 0.77 / +3.9%
tpcds_q25/duckdb:vortex-compact 52054668 / 73029322 / -28.7% 🟢 65728812 / 99567587 / -34.0% 🟢 0.79 / 0.73 / +8.0%
tpcds_q26/duckdb:vortex-compact 30420592 / 48790522 / -37.7% 🟢 38484571 / 64358800 / -40.2% 🟢 0.79 / 0.76 / +4.3%
tpcds_q27/duckdb:vortex-compact 39583630 / 66621600 / -40.6% 🟢 49344025 / 92898441 / -46.9% 🟢 0.80 / 0.72 / +11.9%
tpcds_q28/duckdb:vortex-compact 15559187 / 22653366 / -31.3% 🟢 22834761 / 41801917 / -45.4% 🟢 0.68 / 0.54 / +25.7%
tpcds_q29/duckdb:vortex-compact 54331998 / 83671895 / -35.1% 🟢 69832495 / 106921773 / -34.7% 🟢 0.78 / 0.78 / -0.6%
tpcds_q30/duckdb:vortex-compact 27679862 / 36513065 / -24.2% 🟢 38560708 / 59616757 / -35.3% 🟢 0.72 / 0.61 / +17.2%
tpcds_q31/duckdb:vortex-compact 29120794 / 43401768 / -32.9% 🟢 40562226 / 66737031 / -39.2% 🟢 0.72 / 0.65 / +10.4%
tpcds_q32/duckdb:vortex-compact 17962452 / 25546662 / -29.7% 🟢 27255335 / 42835094 / -36.4% 🟢 0.66 / 0.60 / +10.5%
tpcds_q33/duckdb:vortex-compact 28066445 / 44702651 / -37.2% 🟢 42979076 / 74107589 / -42.0% 🟢 0.65 / 0.60 / +8.3%
tpcds_q34/duckdb:vortex-compact 26981572 / 46536179 / -42.0% 🟢 36932186 / 66274240 / -44.3% 🟢 0.73 / 0.70 / +4.0%
tpcds_q35/duckdb:vortex-compact 77191679 / 113667432 / -32.1% 🟢 91653964 / 152538965 / -39.9% 🟢 0.84 / 0.75 / +13.0%
tpcds_q36/duckdb:vortex-compact 27568300 / 43568184 / -36.7% 🟢 40682720 / 70514172 / -42.3% 🟢 0.68 / 0.62 / +9.7%
tpcds_q37/duckdb:vortex-compact 18531358 / 27886655 / -33.5% 🟢 29025155 / 54769460 / -47.0% 🟢 0.64 / 0.51 / +25.4%
tpcds_q38/duckdb:vortex-compact 33037420 / 50405074 / -34.5% 🟢 40642485 / 71648026 / -43.3% 🟢 0.81 / 0.70 / +15.5%
tpcds_q39/duckdb:vortex-compact 25008917 / 37885867 / -34.0% 🟢 37396187 / 80654210 / -53.6% 🟢 0.67 / 0.47 / +42.4%
tpcds_q40/duckdb:vortex-compact 20577435 / 26822914 / -23.3% 🟢 27362475 / 45269527 / -39.6% 🟢 0.75 / 0.59 / +26.9%
tpcds_q41/duckdb:vortex-compact 9753140 / 11283062 / -13.6% 🟢 15831713 / 23248520 / -31.9% 🟢 0.62 / 0.49 / +26.9%
tpcds_q42/duckdb:vortex-compact 14130166 / 22330719 / -36.7% 🟢 27809106 / 44647744 / -37.7% 🟢 0.51 / 0.50 / +1.6%
tpcds_q43/duckdb:vortex-compact 19482259 / 35572546 / -45.2% 🟢 29498276 / 52086431 / -43.4% 🟢 0.66 / 0.68 / -3.3%
tpcds_q44/duckdb:vortex-compact 28253584 / 34954679 / -19.2% 🟢 42428855 / 72222078 / -41.3% 🟢 0.67 / 0.48 / +37.6%
tpcds_q45/duckdb:vortex-compact 34514650 / 42943274 / -19.6% 🟢 41257643 / 58727165 / -29.7% 🟢 0.84 / 0.73 / +14.4%
tpcds_q46/duckdb:vortex-compact 35877312 / 60872640 / -41.1% 🟢 47589908 / 77876379 / -38.9% 🟢 0.75 / 0.78 / -3.6%
tpcds_q47/duckdb:vortex-compact 44937468 / 65624854 / -31.5% 🟢 58038448 / 99705822 / -41.8% 🟢 0.77 / 0.66 / +17.6%
tpcds_q48/duckdb:vortex-compact 40083804 / 69658153 / -42.5% 🟢 54750617 / 96224900 / -43.1% 🟢 0.73 / 0.72 / +1.1%
tpcds_q49/duckdb:vortex-compact 34301306 / 47435458 / -27.7% 🟢 48169212 / 86064765 / -44.0% 🟢 0.71 / 0.55 / +29.2%
tpcds_q50/duckdb:vortex-compact 33167306 / 54534706 / -39.2% 🟢 45713374 / 83352328 / -45.2% 🟢 0.73 / 0.65 / +10.9%
tpcds_q51/duckdb:vortex-compact 103576519 / 99673422 / +3.9% 83072317 / 147910322 / -43.8% 🟢 1.25 / 0.67 / +85.0%
tpcds_q52/duckdb:vortex-compact 14365239 / 25605063 / -43.9% 🟢 27358127 / 48553041 / -43.7% 🟢 0.53 / 0.53 / -0.4%
tpcds_q53/duckdb:vortex-compact 26730060 / 41214035 / -35.1% 🟢 38713813 / 64801063 / -40.3% 🟢 0.69 / 0.64 / +8.6%
tpcds_q54/duckdb:vortex-compact 32482432 / 45686226 / -28.9% 🟢 42112253 / 75024758 / -43.9% 🟢 0.77 / 0.61 / +26.7%
tpcds_q55/duckdb:vortex-compact 14338029 / 24519526 / -41.5% 🟢 27724109 / 46914416 / -40.9% 🟢 0.52 / 0.52 / -1.0%
tpcds_q56/duckdb:vortex-compact 28639308 / 45660727 / -37.3% 🟢 41839427 / 75529707 / -44.6% 🟢 0.68 / 0.60 / +13.2%
tpcds_q57/duckdb:vortex-compact 35308587 / 46684615 / -24.4% 🟢 45607873 / 65870944 / -30.8% 🟢 0.77 / 0.71 / +9.2%
tpcds_q58/duckdb:vortex-compact 25935504 / 38324066 / -32.3% 🟢 38618314 / 67941616 / -43.2% 🟢 0.67 / 0.56 / +19.1%
tpcds_q59/duckdb:vortex-compact 27659740 / 48422910 / -42.9% 🟢 37136405 / 68260984 / -45.6% 🟢 0.74 / 0.71 / +5.0%
tpcds_q60/duckdb:vortex-compact 29518074 / 45100247 / -34.6% 🟢 42671014 / 78856329 / -45.9% 🟢 0.69 / 0.57 / +21.0%
tpcds_q61/duckdb:vortex-compact 51843392 / 64346640 / -19.4% 🟢 65440820 / 105042440 / -37.7% 🟢 0.79 / 0.61 / +29.3%
tpcds_q62/duckdb:vortex-compact 14317941 / 22863093 / -37.4% 🟢 21568058 / 37031818 / -41.8% 🟢 0.66 / 0.62 / +7.5%
tpcds_q63/duckdb:vortex-compact 26513805 / 38382917 / -30.9% 🟢 37580536 / 62364311 / -39.7% 🟢 0.71 / 0.62 / +14.6%
tpcds_q64/duckdb:vortex-compact 102985924 / 164563110 / -37.4% 🟢 121981661 / 195594775 / -37.6% 🟢 0.84 / 0.84 / +0.3%
tpcds_q65/duckdb:vortex-compact 22232942 / 36288138 / -38.7% 🟢 33685084 / 60192382 / -44.0% 🟢 0.66 / 0.60 / +9.5%
tpcds_q66/duckdb:vortex-compact 31212618 / 44466303 / -29.8% 🟢 39562279 / 72136216 / -45.2% 🟢 0.79 / 0.62 / +28.0%
tpcds_q67/duckdb:vortex-compact 92451222 / 138504119 / -33.3% 🟢 112035344 / 170295507 / -34.2% 🟢 0.83 / 0.81 / +1.5%
tpcds_q68/duckdb:vortex-compact 35795263 / 61294260 / -41.6% 🟢 49663034 / 84937145 / -41.5% 🟢 0.72 / 0.72 / -0.1%
tpcds_q69/duckdb:vortex-compact 56231934 / 89248488 / -37.0% 🟢 70216898 / 120441723 / -41.7% 🟢 0.80 / 0.74 / +8.1%
tpcds_q70/duckdb:vortex-compact 37861110 / 65148430 / -41.9% 🟢 47908308 / 86713842 / -44.8% 🟢 0.79 / 0.75 / +5.2%
tpcds_q71/duckdb:vortex-compact 23535676 / 40028010 / -41.2% 🟢 35605267 / 65050522 / -45.3% 🟢 0.66 / 0.62 / +7.4%
tpcds_q72/duckdb:vortex-compact 101904464 / 163372958 / -37.6% 🟢 116591350 / 220287249 / -47.1% 🟢 0.87 / 0.74 / +17.9%
tpcds_q73/duckdb:vortex-compact 26205996 / 44342456 / -40.9% 🟢 35817653 / 68900124 / -48.0% 🟢 0.73 / 0.64 / +13.7%
tpcds_q74/duckdb:vortex-compact 39846568 / 58854564 / -32.3% 🟢 51091141 / 91965787 / -44.4% 🟢 0.78 / 0.64 / +21.9%
tpcds_q75/duckdb:vortex-compact 40797863 / 66242369 / -38.4% 🟢 53196831 / 104498629 / -49.1% 🟢 0.77 / 0.63 / +21.0%
tpcds_q76/duckdb:vortex-compact 25023216 / 30811017 / -18.8% 🟢 41033944 / 65431616 / -37.3% 🟢 0.61 / 0.47 / +29.5%
tpcds_q77/duckdb:vortex-compact 27602380 / 43946222 / -37.2% 🟢 40100101 / 76742117 / -47.7% 🟢 0.69 / 0.57 / +20.2%
tpcds_q78/duckdb:vortex-compact 53271644 / 84218764 / -36.7% 🟢 62498508 / 108339833 / -42.3% 🟢 0.85 / 0.78 / +9.6%
tpcds_q79/duckdb:vortex-compact 46271312 / 84391578 / -45.2% 🟢 56765809 / 104465776 / -45.7% 🟢 0.82 / 0.81 / +0.9%
tpcds_q80/duckdb:vortex-compact 45005477 / 71225884 / -36.8% 🟢 59770775 / 96310172 / -37.9% 🟢 0.75 / 0.74 / +1.8%
tpcds_q81/duckdb:vortex-compact 30138438 / 38532152 / -21.8% 🟢 42400244 / 64157489 / -33.9% 🟢 0.71 / 0.60 / +18.4%
tpcds_q82/duckdb:vortex-compact 43545046 / 60314137 / -27.8% 🟢 50990701 / 82305967 / -38.0% 🟢 0.85 / 0.73 / +16.5%
tpcds_q83/duckdb:vortex-compact 37002119 / 46282694 / -20.1% 🟢 40922755 / 61230948 / -33.2% 🟢 0.90 / 0.76 / +19.6%
tpcds_q84/duckdb:vortex-compact 17451790 / 24846013 / -29.8% 🟢 25516794 / 44375212 / -42.5% 🟢 0.68 / 0.56 / +22.2%
tpcds_q85/duckdb:vortex-compact 51007294 / 69572058 / -26.7% 🟢 64764438 / 90436185 / -28.4% 🟢 0.79 / 0.77 / +2.4%
tpcds_q86/duckdb:vortex-compact 15316954 / 21392048 / -28.4% 🟢 23800828 / 37729324 / -36.9% 🟢 0.64 / 0.57 / +13.5%
tpcds_q87/duckdb:vortex-compact 35352328 / 52319462 / -32.4% 🟢 42637038 / 72654581 / -41.3% 🟢 0.83 / 0.72 / +15.1%
tpcds_q88/duckdb:vortex-compact 101307624 / 121448417 / -16.6% 🟢 113542841 / 187255237 / -39.4% 🟢 0.89 / 0.65 / +37.6%
tpcds_q89/duckdb:vortex-compact 25834210 / 40833436 / -36.7% 🟢 36144587 / 67046980 / -46.1% 🟢 0.71 / 0.61 / +17.4%
tpcds_q90/duckdb:vortex-compact 14283106 / 21486482 / -33.5% 🟢 19173393 / 41180758 / -53.4% 🟢 0.74 / 0.52 / +42.8%
tpcds_q91/duckdb:vortex-compact 39323308 / 66693722 / -41.0% 🟢 49189598 / 84709447 / -41.9% 🟢 0.80 / 0.79 / +1.5%
tpcds_q92/duckdb:vortex-compact 26004275 / 35798339 / -27.4% 🟢 34806317 / 52217243 / -33.3% 🟢 0.75 / 0.69 / +9.0%
tpcds_q93/duckdb:vortex-compact 22661483 / 36639774 / -38.2% 🟢 32840684 / 64624580 / -49.2% 🟢 0.69 / 0.57 / +21.7%
tpcds_q94/duckdb:vortex-compact 26391518 / 39369948 / -33.0% 🟢 39120810 / 61954773 / -36.9% 🟢 0.67 / 0.64 / +6.2%
tpcds_q95/duckdb:vortex-compact 95517362 / 153442106 / -37.8% 🟢 103506507 / 173860328 / -40.5% 🟢 0.92 / 0.88 / +4.6%
tpcds_q96/duckdb:vortex-compact 20694050 / 34706532 / -40.4% 🟢 29798391 / 57264252 / -48.0% 🟢 0.69 / 0.61 / +14.6%
tpcds_q97/duckdb:vortex-compact 31724457 / 47749086 / -33.6% 🟢 41016357 / 64752815 / -36.7% 🟢 0.77 / 0.74 / +4.9%
tpcds_q98/duckdb:vortex-compact 22069180 / 31988165 / -31.0% 🟢 35136679 / 57028309 / -38.4% 🟢 0.63 / 0.56 / +12.0%
tpcds_q99/duckdb:vortex-compact 20828704 / 2757862 / -24.5% 🟢 27869003 / 45665276 / -39.0% 🟢 0.75 / 0.60 / +23.8%
duckdb / parquet / ns (0.878x ✅, 61↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpcds_q01/duckdb:parquet 30120583 / 32317048 / -6.8% 40563262 / 45787749 / -11.4% 🟢 0.74 / 0.71 / +5.2%
tpcds_q02/duckdb:parquet 18898218 / 22387552 / -15.6% 🟢 25936211 / 32654508 / -20.6% 🟢 0.73 / 0.69 / +6.3%
tpcds_q03/duckdb:parquet 11385431 / 12423147 / -8.4% 21302798 / 22395567 / -4.9% 0.53 / 0.55 / -3.7%
tpcds_q04/duckdb:parquet 162877743 / 193572593 / -15.9% 🟢 174816896 / 205153235 / -14.8% 🟢 0.93 / 0.94 / -1.3%
tpcds_q05/duckdb:parquet 28311276 / 32333356 / -12.4% 🟢 37475714 / 42716778 / -12.3% 🟢 0.76 / 0.76 / -0.2%
tpcds_q06/duckdb:parquet 33443684 / 35400502 / -5.5% 46473270 / 54301484 / -14.4% 🟢 0.72 / 0.65 / +10.4%
tpcds_q07/duckdb:parquet 19349647 / 21569910 / -10.3% 🟢 35702635 / 35803823 / -0.3% 0.54 / 0.60 / -10.0%
tpcds_q08/duckdb:parquet 32146168 / 33510936 / -4.1% 42068560 / 45015843 / -6.5% 0.76 / 0.74 / +2.6%
tpcds_q09/duckdb:parquet 28282554 / 31557464 / -10.4% 🟢 34853001 / 38210835 / -8.8% 0.81 / 0.83 / -1.7%
tpcds_q10/duckdb:parquet 35574474 / 39748214 / -10.5% 🟢 49382889 / 60132366 / -17.9% 🟢 0.72 / 0.66 / +9.0%
tpcds_q11/duckdb:parquet 90801525 / 104480510 / -13.1% 🟢 101536668 / 114549369 / -11.4% 🟢 0.89 / 0.91 / -2.0%
tpcds_q12/duckdb:parquet 17055491 / 17866056 / -4.5% 24017161 / 27018869 / -11.1% 🟢 0.71 / 0.66 / +7.4%
tpcds_q13/duckdb:parquet 30177206 / 34321696 / -12.1% 🟢 43852587 / 50240596 / -12.7% 🟢 0.69 / 0.68 / +0.7%
tpcds_q14/duckdb:parquet 80484992 / 103908720 / -22.5% 🟢 91759330 / 117194473 / -21.7% 🟢 0.88 / 0.89 / -1.1%
tpcds_q15/duckdb:parquet 32628234 / 35042373 / -6.9% 40179216 / 47006915 / -14.5% 🟢 0.81 / 0.75 / +8.9%
tpcds_q16/duckdb:parquet 25543488 / 27309796 / -6.5% 35157886 / 43162966 / -18.5% 🟢 0.73 / 0.63 / +14.8%
tpcds_q17/duckdb:parquet 42045308 / 44432462 / -5.4% 52644790 / 56406514 / -6.7% 0.80 / 0.79 / +1.4%
tpcds_q18/duckdb:parquet 49392113 / 57341868 / -13.9% 🟢 63474381 / 76373703 / -16.9% 🟢 0.78 / 0.75 / +3.6%
tpcds_q19/duckdb:parquet 31793953 / 33612052 / -5.4% 43750061 / 46522084 / -6.0% 0.73 / 0.72 / +0.6%
tpcds_q20/duckdb:parquet 18002566 / 19397260 / -7.2% 25034149 / 34662958 / -27.8% 🟢 0.72 / 0.56 / +28.5%
tpcds_q21/duckdb:parquet 12286531 / 14512242 / -15.3% 🟢 18109321 / 30123638 / -39.9% 🟢 0.68 / 0.48 / +40.8%
tpcds_q22/duckdb:parquet 46279526 / 56272406 / -17.8% 🟢 51615130 / 77079806 / -33.0% 🟢 0.90 / 0.73 / +22.8%
tpcds_q23/duckdb:parquet 53706946 / 68216878 / -21.3% 🟢 63062532 / 81066947 / -22.2% 🟢 0.85 / 0.84 / +1.2%
tpcds_q24/duckdb:parquet 44571829 / 49166633 / -9.3% 63315227 / 69605168 / -9.0% 0.70 / 0.71 / -0.3%
tpcds_q25/duckdb:parquet 39217026 / 41846615 / -6.3% 51297507 / 57210384 / -10.3% 🟢 0.76 / 0.73 / +4.5%
tpcds_q26/duckdb:parquet 39811256 / 43598212 / -8.7% 49680331 / 55072272 / -9.8% 0.80 / 0.79 / +1.2%
tpcds_q27/duckdb:parquet 40772262 / 56956951 / -28.4% 🟢 58050712 / 69969611 / -17.0% 🟢 0.70 / 0.81 / -13.7%
tpcds_q28/duckdb:parquet 24371176 / 28366980 / -14.1% 🟢 33045894 / 38330724 / -13.8% 🟢 0.74 / 0.74 / -0.3%
tpcds_q29/duckdb:parquet 39851234 / 42725754 / -6.7% 50580028 / 56609418 / -10.7% 🟢 0.79 / 0.75 / +4.4%
tpcds_q30/duckdb:parquet 40905944 / 42844947 / -4.5% 52624533 / 61717590 / -14.7% 🟢 0.78 / 0.69 / +12.0%
tpcds_q31/duckdb:parquet 26049029 / 29312290 / -11.1% 🟢 32964948 / 41727132 / -21.0% 🟢 0.79 / 0.70 / +12.5%
tpcds_q32/duckdb:parquet 14232374 / 15905922 / -10.5% 🟢 22240684 / 27685364 / -19.7% 🟢 0.64 / 0.57 / +11.4%
tpcds_q33/duckdb:parquet 21389557 / 24187074 / -11.6% 🟢 34391236 / 38541041 / -10.8% 🟢 0.62 / 0.63 / -0.9%
tpcds_q34/duckdb:parquet 20612225 / 22234396 / -7.3% 31010543 / 34299340 / -9.6% 0.66 / 0.65 / +2.5%
tpcds_q35/duckdb:parquet 60025916 / 66190742 / -9.3% 73039491 / 85580146 / -14.7% 🟢 0.82 / 0.77 / +6.3%
tpcds_q36/duckdb:parquet 19599692 / 22582320 / -13.2% 🟢 32585681 / 39590146 / -17.7% 🟢 0.60 / 0.57 / +5.4%
tpcds_q37/duckdb:parquet 14868120 / 17206177 / -13.6% 🟢 21487278 / 32617002 / -34.1% 🟢 0.69 / 0.53 / +31.2%
tpcds_q38/duckdb:parquet 34359573 / 40424984 / -15.0% 🟢 39285002 / 52581747 / -25.3% 🟢 0.87 / 0.77 / +13.8%
tpcds_q39/duckdb:parquet 32154622 / 37453686 / -14.1% 🟢 38236247 / 51187213 / -25.3% 🟢 0.84 / 0.73 / +14.9%
tpcds_q40/duckdb:parquet 20952122 / 23002736 / -8.9% 27411639 / 33462212 / -18.1% 🟢 0.76 / 0.69 / +11.2%
tpcds_q41/duckdb:parquet 9609198 / 10453702 / -8.1% 16631063 / 25045262 / -33.6% 🟢 0.58 / 0.42 / +38.4%
tpcds_q42/duckdb:parquet 10841621 / 12091169 / -10.3% 🟢 21196871 / 21267686 / -0.3% 0.51 / 0.57 / -10.0%
tpcds_q43/duckdb:parquet 13454320 / 14759287 / -8.8% 22306894 / 26443365 / -15.6% 🟢 0.60 / 0.56 / +8.1%
tpcds_q44/duckdb:parquet 22788138 / 25846535 / -11.8% 🟢 30444347 / 38994137 / -21.9% 🟢 0.75 / 0.66 / +12.9%
tpcds_q45/duckdb:parquet 31086632 / 32793370 / -5.2% 39099532 / 43778924 / -10.7% 🟢 0.80 / 0.75 / +6.1%
tpcds_q46/duckdb:parquet 46791376 / 52293226 / -10.5% 🟢 62765725 / 65115599 / -3.6% 0.75 / 0.80 / -7.2%
tpcds_q47/duckdb:parquet 40835432 / 50237382 / -18.7% 🟢 53024955 / 63094354 / -16.0% 🟢 0.77 / 0.80 / -3.3%
tpcds_q48/duckdb:parquet 26543154 / 30538212 / -13.1% 🟢 40015989 / 49728580 / -19.5% 🟢 0.66 / 0.61 / +8.0%
tpcds_q49/duckdb:parquet 24499572 / 29271002 / -16.3% 🟢 41358733 / 43525693 / -5.0% 0.59 / 0.67 / -11.9%
tpcds_q50/duckdb:parquet 22513634 / 25224469 / -10.7% 🟢 34315482 / 36598459 / -6.2% 0.66 / 0.69 / -4.8%
tpcds_q51/duckdb:parquet 70218394 / 94522557 / -25.7% 🟢 81154336 / 145363033 / -44.2% 🟢 0.87 / 0.65 / +33.1%
tpcds_q52/duckdb:parquet 11005956 / 13726322 / -19.8% 🟢 21431404 / 25019564 / -14.3% 🟢 0.51 / 0.55 / -6.4%
tpcds_q53/duckdb:parquet 15100040 / 16391856 / -7.9% 25914740 / 29699998 / -12.7% 🟢 0.58 / 0.55 / +5.6%
tpcds_q54/duckdb:parquet 26455283 / 30488630 / -13.2% 🟢 38932245 / 46010653 / -15.4% 🟢 0.68 / 0.66 / +2.5%
tpcds_q55/duckdb:parquet 11005664 / 13718776 / -19.8% 🟢 21622241 / 23860243 / -9.4% 0.51 / 0.57 / -11.5%
tpcds_q56/duckdb:parquet 21575056 / 24926415 / -13.4% 🟢 34184349 / 38807497 / -11.9% 🟢 0.63 / 0.64 / -1.7%
tpcds_q57/duckdb:parquet 38282261 / 43980511 / -13.0% 🟢 45640994 / 56110160 / -18.7% 🟢 0.84 / 0.78 / +7.0%
tpcds_q58/duckdb:parquet 24111778 / 26757778 / -9.9% 35954981 / 39690022 / -9.4% 0.67 / 0.67 / -0.5%
tpcds_q59/duckdb:parquet 25387366 / 29913732 / -15.1% 🟢 34090054 / 53255485 / -36.0% 🟢 0.74 / 0.56 / +32.6%
tpcds_q60/duckdb:parquet 22461460 / 27113364 / -17.2% 🟢 35258960 / 40888549 / -13.8% 🟢 0.64 / 0.66 / -3.9%
tpcds_q61/duckdb:parquet 27263530 / 32622806 / -16.4% 🟢 39107491 / 42551764 / -8.1% 0.70 / 0.77 / -9.1%
tpcds_q62/duckdb:parquet 14125933 / 15097566 / -6.4% 20740539 / 22675389 / -8.5% 0.68 / 0.67 / +2.3%
tpcds_q63/duckdb:parquet 14492316 / 17508506 / -17.2% 🟢 26125408 / 29944684 / -12.8% 🟢 0.55 / 0.58 / -5.1%
tpcds_q64/duckdb:parquet 65887210 / 80027398 / -17.7% 🟢 89263081 / 104850566 / -14.9% 🟢 0.74 / 0.76 / -3.3%
tpcds_q65/duckdb:parquet 19134986 / 21766151 / -12.1% 🟢 30055430 / 36327491 / -17.3% 🟢 0.64 / 0.60 / +6.3%
tpcds_q66/duckdb:parquet 31829712 / 35802368 / -11.1% 🟢 40708083 / 46992420 / -13.4% 🟢 0.78 / 0.76 / +2.6%
tpcds_q67/duckdb:parquet 103705171 / 124523440 / -16.7% 🟢 119467457 / 133784186 / -10.7% 🟢 0.87 / 0.93 / -6.7%
tpcds_q68/duckdb:parquet 36398138 / 39740302 / -8.4% 53970068 / 58778944 / -8.2% 0.67 / 0.68 / -0.2%
tpcds_q69/duckdb:parquet 36651397 / 41487183 / -11.7% 🟢 50480041 / 63709188 / -20.8% 🟢 0.73 / 0.65 / +11.5%
tpcds_q70/duckdb:parquet 19068892 / 28277714 / -32.6% 🟢 28526773 / 44382816 / -35.7% 🟢 0.67 / 0.64 / +4.9%
tpcds_q71/duckdb:parquet 19082986 / 21573095 / -11.5% 🟢 31263475 / 36919352 / -15.3% 🟢 0.61 / 0.58 / +4.5%
tpcds_q72/duckdb:parquet 104833482 / 131588069 / -20.3% 🟢 111779500 / 132847216 / -15.9% 🟢 0.94 / 0.99 / -5.3%
tpcds_q73/duckdb:parquet 17317940 / 18804694 / -7.9% 27121899 / 32284513 / -16.0% 🟢 0.64 / 0.58 / +9.6%
tpcds_q74/duckdb:parquet 131068605 / 146195980 / -10.3% 🟢 139975789 / 162246430 / -13.7% 🟢 0.94 / 0.90 / +3.9%
tpcds_q75/duckdb:parquet 52024308 / 60051390 / -13.4% 🟢 70220525 / 79909749 / -12.1% 🟢 0.74 / 0.75 / -1.4%
tpcds_q76/duckdb:parquet 17587596 / 19793437 / -11.1% 🟢 33034744 / 33651180 / -1.8% 0.53 / 0.59 / -9.5%
tpcds_q77/duckdb:parquet 22165296 / 27252595 / -18.7% 🟢 35041005 / 40019640 / -12.4% 🟢 0.63 / 0.68 / -7.1%
tpcds_q78/duckdb:parquet 64071390 / 77733072 / -17.6% 🟢 71057370 / 91046656 / -22.0% 🟢 0.90 / 0.85 / +5.6%
tpcds_q79/duckdb:parquet 25986612 / 30835768 / -15.7% 🟢 40625959 / 46651101 / -12.9% 🟢 0.64 / 0.66 / -3.2%
tpcds_q80/duckdb:parquet 40437944 / 49440260 / -18.2% 🟢 56109182 / 60474602 / -7.2% 0.72 / 0.82 / -11.8%
tpcds_q81/duckdb:parquet 37862994 / 41213694 / -8.1% 48787604 / 57270252 / -14.8% 🟢 0.78 / 0.72 / +7.8%
tpcds_q82/duckdb:parquet 16331904 / 19119628 / -14.6% 🟢 27730415 / 32688646 / -15.2% 🟢 0.59 / 0.58 / +0.7%
tpcds_q83/duckdb:parquet 21018970 / 22309993 / -5.8% 28907565 / 31576765 / -8.5% 0.73 / 0.71 / +2.9%
tpcds_q84/duckdb:parquet 22253744 / 23979829 / -7.2% 29768116 / 33166105 / -10.2% 🟢 0.75 / 0.72 / +3.4%
tpcds_q85/duckdb:parquet 45648572 / 48842469 / -6.5% 59251386 / 67993691 / -12.9% 🟢 0.77 / 0.72 / +7.3%
tpcds_q86/duckdb:parquet 14445042 / 15019348 / -3.8% 21174610 / 25492717 / -16.9% 🟢 0.68 / 0.59 / +15.8%
tpcds_q87/duckdb:parquet 36165596 / 44773684 / -19.2% 🟢 40385919 / 54424005 / -25.8% 🟢 0.90 / 0.82 / +8.9%
tpcds_q88/duckdb:parquet 40907446 / 43438634 / -5.8% 44334275 / 52636446 / -15.8% 🟢 0.92 / 0.83 / +11.8%
tpcds_q89/duckdb:parquet 16467664 / 19270080 / -14.5% 🟢 28141854 / 34266691 / -17.9% 🟢 0.59 / 0.56 / +4.1%
tpcds_q90/duckdb:parquet 9190980 / 9834613 / -6.5% 14136294 / 20778461 / -32.0% 🟢 0.65 / 0.47 / +37.4%
tpcds_q91/duckdb:parquet 26914824 / 28013814 / -3.9% 37659679 / 41251618 / -8.7% 0.71 / 0.68 / +5.2%
tpcds_q92/duckdb:parquet 14880864 / 15636914 / -4.8% 23801242 / 26983352 / -11.8% 🟢 0.63 / 0.58 / +7.9%
tpcds_q93/duckdb:parquet 27588426 / 31286202 / -11.8% 🟢 42060412 / 47133692 / -10.8% 🟢 0.66 / 0.66 / -1.2%
tpcds_q94/duckdb:parquet 20594612 / 21580781 / -4.6% 31485059 / 36921525 / -14.7% 🟢 0.65 / 0.58 / +11.9%
tpcds_q95/duckdb:parquet 112488488 / 139546692 / -19.4% 🟢 117863379 / 155037704 / -24.0% 🟢 0.95 / 0.90 / +6.0%
tpcds_q96/duckdb:parquet 9082878 / 9605488 / -5.4% 14983609 / 17237953 / -13.1% 🟢 0.61 / 0.56 / +8.8%
tpcds_q97/duckdb:parquet 33432788 / 40025908 / -16.5% 🟢 41275900 / 49616613 / -16.8% 🟢 0.81 / 0.81 / +0.4%
tpcds_q98/duckdb:parquet 19235750 / 22165671 / -13.2% 🟢 30325833 / 33250491 / -8.8% 0.63 / 0.67 / -4.8%
tpcds_q99/duckdb:parquet 21736186 / 24022795 / -9.5% 27719585 / 31859977 / -13.0% 🟢 0.78 / 0.75 / +4.0%
File Size Changes (3 files changed, -0.0% overall, 1↑ 2↓)
File Scale Format Base HEAD Change %
catalog_page.vortex 1.0 vortex-file-compressed 545.95 KB 546.16 KB +208 B +0.0%
customer.vortex 1.0 vortex-file-compressed 4.18 MB 4.18 MB 1.03 KB -0.0%
date_dim.vortex 1.0 vortex-file-compressed 576.08 KB 551.28 KB 24.80 KB -4.3%

Totals:

  • vortex-compact: 207.80 MB → 207.80 MB (0.0%)
  • vortex-file-compressed: 265.99 MB → 265.96 MB (-0.0%)

@github-actions

github-actions Bot commented Sep 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmarks: FineWeb S3 📖

Commits: PR dd8b5948 vs base 792cef74
Verdict: Likely regression (environment too noisy confidence)
Attributed Vortex impact: +146.8%
Engines: DataFusion No clear signal (+26.7%, environment too noisy confidence) · DuckDB Likely regression (+381.0%, environment too noisy confidence)
Vortex (geomean): hot 1.119x ➖ · cold 1.082x ➖
Parquet (geomean): hot 0.453x ✅ · cold 1.083x ➖
Shifts: Parquet (control) -54.7% · Median polish +8.5%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Table cells: Each cell reads PR / base / %diff. The hot and cold columns mark a change as 🔴 slower or 🟢 faster once it clears this suite's threshold; anything smaller is noise here and is left unmarked, as is hot/cold, because a shift in the warm-up ratio is not a win or a loss by itself.
datafusion / vortex-file-compressed / ns (1.242x ➖, 0↑ 3↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
fineweb_q00/datafusion:vortex-file-compressed 41027430 / 33934058 / +20.9% 160240513 / 129923564 / +23.3% 0.26 / 0.26 / -2.0%
fineweb_q01/datafusion:vortex-file-compressed 396593738 / 415839582 / -4.6% 882385000 / 808560428 / +9.1% 0.45 / 0.51 / -12.6%
fineweb_q02/datafusion:vortex-file-compressed 434569848 / 403394866 / +7.7% 978652457 / 444038176 / +120.4% 🔴 0.44 / 0.91 / -51.1%
fineweb_q03/datafusion:vortex-file-compressed 1172301549 / 620356512 / +89.0% 🔴 1652267381 / 724251726 / +128.1% 🔴 0.71 / 0.86 / -17.2%
fineweb_q04/datafusion:vortex-file-compressed 517185425 / 646580926 / -20.0% 563808540 / 504276608 / +11.8% 0.92 / 1.28 / -28.5%
fineweb_q05/datafusion:vortex-file-compressed 507372766 / 582051767 / -12.8% 496904830 / 560186176 / -11.3% 1.02 / 1.04 / -1.7%
fineweb_q06/datafusion:vortex-file-compressed 1280402253 / 659637183 / +94.1% 🔴 893689446 / 692427688 / +29.1% 1.43 / 0.95 / +50.4%
fineweb_q07/datafusion:vortex-file-compressed 570805966 / 511155074 / +11.7% 522550646 / 495198302 / +5.5% 1.09 / 1.03 / +5.8%
fineweb_q08/datafusion:vortex-file-compressed 433386332 / 218453506 / +98.4% 🔴 567024848 / 286193712 / +98.1% 🔴 0.76 / 0.76 / +0.1%
datafusion / vortex-compact / ns (1.059x ➖, 2↑ 3↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
fineweb_q00/datafusion:vortex-compact 59415044 / 37244654 / +59.5% 🔴 145705266 / 118255988 / +23.2% 0.41 / 0.31 / +29.5%
fineweb_q01/datafusion:vortex-compact 386736930 / 455936358 / -15.2% 822631422 / 660627835 / +24.5% 0.47 / 0.69 / -31.9%
fineweb_q02/datafusion:vortex-compact 506197959 / 491970042 / +2.9% 797193052 / 558304222 / +42.8% 🔴 0.63 / 0.88 / -27.9%
fineweb_q03/datafusion:vortex-compact 645839000 / 928472438 / -30.4% 🟢 1771665083 / 1675463824 / +5.7% 0.36 / 0.55 / -34.2%
fineweb_q04/datafusion:vortex-compact 635995263 / 996813832 / -36.2% 🟢 651284046 / 799335965 / -18.5% 0.98 / 1.25 / -21.7%
fineweb_q05/datafusion:vortex-compact 1321420699 / 831387397 / +58.9% 🔴 1151051438 / 781615802 / +47.3% 🔴 1.15 / 1.06 / +7.9%
fineweb_q06/datafusion:vortex-compact 798376454 / 754163160 / +5.9% 776297614 / 782096511 / -0.7% 1.03 / 0.96 / +6.7%
fineweb_q07/datafusion:vortex-compact 840564640 / 614085138 / +36.9% 🔴 631988924 / 537871762 / +17.5% 1.33 / 1.14 / +16.5%
fineweb_q08/datafusion:vortex-compact 216410148 / 183470302 / +18.0% 201003378 / 223323851 / -10.0% 1.08 / 0.82 / +31.1%
datafusion / parquet / ns (0.906x ➖, 1↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
fineweb_q00/datafusion:parquet 580483644 / 693755275 / -16.3% 1130257452 / 1132309050 / -0.2% 0.51 / 0.61 / -16.2%
fineweb_q01/datafusion:parquet 966265461 / 1211335513 / -20.2% 1040461121 / 1013795694 / +2.6% 0.93 / 1.19 / -22.3%
fineweb_q02/datafusion:parquet 920469007 / 1315500536 / -30.0% 🟢 991390215 / 1471278474 / -32.6% 🟢 0.93 / 0.89 / +3.8%
fineweb_q03/datafusion:parquet 995109606 / 1059150416 / -6.0% 1015424838 / 883248586 / +15.0% 0.98 / 1.20 / -18.3%
fineweb_q04/datafusion:parquet 881389130 / 1020456081 / -13.6% 1003243248 / 1088755474 / -7.9% 0.88 / 0.94 / -6.3%
fineweb_q05/datafusion:parquet 1046552222 / 990891421 / +5.6% 1102549860 / 1153698779 / -4.4% 0.95 / 0.86 / +10.5%
fineweb_q06/datafusion:parquet 973018545 / 922590704 / +5.5% 901542683 / 1352734992 / -33.4% 🟢 1.08 / 0.68 / +58.2%
fineweb_q07/datafusion:parquet 921953940 / 974564193 / -5.4% 901958122 / 1356288447 / -33.5% 🟢 1.02 / 0.72 / +42.3%
fineweb_q08/datafusion:parquet 998718104 / 974290372 / +2.5% 944888391 / 1106100674 / -14.6% 1.06 / 0.88 / +20.0%
duckdb / vortex-file-compressed / ns (1.123x ➖, 0↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
fineweb_q00/duckdb:vortex-file-compressed 76737615 / 75133720 / +2.1% 45938931 / 117800435 / -61.0% 🟢 1.67 / 0.64 / +161.9%
fineweb_q01/duckdb:vortex-file-compressed 545985068 / 544560636 / +0.3% 666052672 / 801309229 / -16.9% 0.82 / 0.68 / +20.6%
fineweb_q02/duckdb:vortex-file-compressed 729109920 / 684831795 / +6.5% 728256855 / 683498879 / +6.5% 1.00 / 1.00 / -0.1%
fineweb_q03/duckdb:vortex-file-compressed 1694534332 / 1450214828 / +16.8% 1864389357 / 1625824985 / +14.7% 0.91 / 0.89 / +1.9%
fineweb_q04/duckdb:vortex-file-compressed 1819924750 / 1605175312 / +13.4% 1935892547 / 1639535632 / +18.1% 0.94 / 0.98 / -4.0%
fineweb_q05/duckdb:vortex-file-compressed 1693063484 / 1516116553 / +11.7% 1700050552 / 1463887236 / +16.1% 1.00 / 1.04 / -3.8%
fineweb_q06/duckdb:vortex-file-compressed 2106063014 / 1686168637 / +24.9% 2016575211 / 1778988574 / +13.4% 1.04 / 0.95 / +10.2%
fineweb_q07/duckdb:vortex-file-compressed 1668081390 / 1431474336 / +16.5% 1814045590 / 1537717933 / +18.0% 0.92 / 0.93 / -1.2%
fineweb_q08/duckdb:vortex-file-compressed 725099259 / 597383380 / +21.4% 730339689 / 689060725 / +6.0% 0.99 / 0.87 / +14.5%
duckdb / vortex-compact / ns (1.062x ➖, 0↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
fineweb_q00/duckdb:vortex-compact 74857662 / 71449682 / +4.8% 57815986 / 143375670 / -59.7% 🟢 1.29 / 0.50 / +159.8%
fineweb_q01/duckdb:vortex-compact 601999020 / 516152037 / +16.6% 755664132 / 1164250212 / -35.1% 🟢 0.80 / 0.44 / +79.7%
fineweb_q02/duckdb:vortex-compact 683546930 / 686325937 / -0.4% 710943229 / 718287887 / -1.0% 0.96 / 0.96 / +0.6%
fineweb_q03/duckdb:vortex-compact 1993021075 / 1955235654 / +1.9% 2075641278 / 2055412168 / +1.0% 0.96 / 0.95 / +0.9%
fineweb_q04/duckdb:vortex-compact 1887390292 / 1808954422 / +4.3% 1988412471 / 1726324394 / +15.2% 0.95 / 1.05 / -9.4%
fineweb_q05/duckdb:vortex-compact 1698575882 / 1667175928 / +1.9% 1779236712 / 1602762853 / +11.0% 0.95 / 1.04 / -8.2%
fineweb_q06/duckdb:vortex-compact 1825118100 / 1694231276 / +7.7% 1768456114 / 1669769043 / +5.9% 1.03 / 1.01 / +1.7%
fineweb_q07/duckdb:vortex-compact 1681538530 / 1543183380 / +9.0% 1695644756 / 1423873488 / +19.1% 0.99 / 1.08 / -8.5%
fineweb_q08/duckdb:vortex-compact 604499556 / 546532566 / +10.6% 611237582 / 622302975 / -1.8% 0.99 / 0.88 / +12.6%
duckdb / parquet / ns (0.227x ✅, 9↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
fineweb_q00/duckdb:parquet 682190714 / 5277657002 / -87.1% 🟢 705541218 / 660981602 / +6.7% 0.97 / 7.98 / -87.9%
fineweb_q01/duckdb:parquet 816027064 / 5320437194 / -84.7% 🟢 822059148 / 668263904 / +23.0% 0.99 / 7.96 / -87.5%
fineweb_q02/duckdb:parquet 815060955 / 5301419810 / -84.6% 🟢 804514739 / 622003383 / +29.3% 1.01 / 8.52 / -88.1%
fineweb_q03/duckdb:parquet 1907748300 / 5345532504 / -64.3% 🟢 1985347506 / 1185093310 / +67.5% 🔴 0.96 / 4.51 / -78.7%
fineweb_q04/duckdb:parquet 1192294314 / 5287844374 / -77.5% 🟢 1128724041 / 948145724 / +19.0% 1.06 / 5.58 / -81.1%
fineweb_q05/duckdb:parquet 1220393369 / 3030363718 / -59.7% 🟢 1235185933 / 981367706 / +25.9% 0.99 / 3.09 / -68.0%
fineweb_q06/duckdb:parquet 2271209188 / 5394625164 / -57.9% 🟢 2310179103 / 1372361154 / +68.3% 🔴 0.98 / 3.93 / -75.0%
fineweb_q07/duckdb:parquet 1567720432 / 5367184224 / -70.8% 🟢 1458632689 / 980449011 / +48.8% 🔴 1.07 / 5.47 / -80.4%
fineweb_q08/duckdb:parquet 693556336 / 5255816708 / -86.8% 🟢 976660615 / 657958542 / +48.4% 🔴 0.71 / 7.99 / -91.1%

@github-actions

github-actions Bot commented Sep 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmarks: Clickbench on NVME 📖

Commits: PR dd8b5948 vs base 792cef74
Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: -2.6%
Engines: DataFusion No clear signal (-3.4%, environment too noisy confidence) · DuckDB No clear signal (-1.9%, environment too noisy confidence)
Vortex (geomean): hot 0.691x ✅ · cold 0.634x ✅
Parquet (geomean): hot 0.710x ✅ · cold 0.784x ✅
Shifts: Parquet (control) -29.0% · Median polish -31.4%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Table cells: Each cell reads PR / base / %diff. The hot and cold columns mark a change as 🔴 slower or 🟢 faster once it clears this suite's threshold; anything smaller is noise here and is left unmarked, as is hot/cold, because a shift in the warm-up ratio is not a win or a loss by itself.
datafusion / vortex-file-compressed / ns (0.679x ✅, 42↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
clickbench_q00/datafusion:vortex-file-compressed 1513208 / 1948896 / -22.4% 🟢 33252523 / 40849926 / -18.6% 🟢 0.05 / 0.05 / -4.6%
clickbench_q01/datafusion:vortex-file-compressed 10084704 / 12074758 / -16.5% 🟢 42478913 / 59988928 / -29.2% 🟢 0.24 / 0.20 / +17.9%
clickbench_q02/datafusion:vortex-file-compressed 19550722 / 25955878 / -24.7% 🟢 54362053 / 76390333 / -28.8% 🟢 0.36 / 0.34 / +5.8%
clickbench_q03/datafusion:vortex-file-compressed 19652048 / 26989758 / -27.2% 🟢 95078329 / 96973641 / -2.0% 0.21 / 0.28 / -25.7%
clickbench_q04/datafusion:vortex-file-compressed 94436193 / 159603173 / -40.8% 🟢 145390469 / 232036568 / -37.3% 🟢 0.65 / 0.69 / -5.6%
clickbench_q05/datafusion:vortex-file-compressed 118921061 / 217357066 / -45.3% 🟢 176575618 / 295619628 / -40.3% 🟢 0.67 / 0.74 / -8.4%
clickbench_q06/datafusion:vortex-file-compressed 1544401 / 1928178 / -19.9% 🟢 33866113 / 41854588 / -19.1% 🟢 0.05 / 0.05 / -1.0%
clickbench_q07/datafusion:vortex-file-compressed 14092134 / 18082204 / -22.1% 🟢 48528556 / 68981679 / -29.7% 🟢 0.29 / 0.26 / +10.8%
clickbench_q08/datafusion:vortex-file-compressed 132053770 / 217125464 / -39.2% 🟢 189868840 / 308794556 / -38.5% 🟢 0.70 / 0.70 / -1.1%
clickbench_q09/datafusion:vortex-file-compressed 244134260 / 352035971 / -30.7% 🟢 303625054 / 410605328 / -26.1% 🟢 0.80 / 0.86 / -6.2%
clickbench_q10/datafusion:vortex-file-compressed 47098616 / 66754855 / -29.4% 🟢 108560118 / 132404332 / -18.0% 🟢 0.43 / 0.50 / -13.9%
clickbench_q11/datafusion:vortex-file-compressed 61256569 / 86041428 / -28.8% 🟢 110842646 / 150413334 / -26.3% 🟢 0.55 / 0.57 / -3.4%
clickbench_q12/datafusion:vortex-file-compressed 114696161 / 192836172 / -40.5% 🟢 170366561 / 279971020 / -39.1% 🟢 0.67 / 0.69 / -2.3%
clickbench_q13/datafusion:vortex-file-compressed 196432616 / 329475628 / -40.4% 🟢 261974951 / 443044644 / -40.9% 🟢 0.75 / 0.74 / +0.8%
clickbench_q14/datafusion:vortex-file-compressed 128101057 / 201807876 / -36.5% 🟢 180228465 / 288242352 / -37.5% 🟢 0.71 / 0.70 / +1.5%
clickbench_q15/datafusion:vortex-file-compressed 109732431 / 184395978 / -40.5% 🟢 160073591 / 262443092 / -39.0% 🟢 0.69 / 0.70 / -2.4%
clickbench_q16/datafusion:vortex-file-compressed 264723574 / 467286492 / -43.3% 🟢 337204850 / 571835307 / -41.0% 🟢 0.79 / 0.82 / -3.9%
clickbench_q17/datafusion:vortex-file-compressed 263955914 / 454539872 / -41.9% 🟢 327005950 / 590144012 / -44.6% 🟢 0.81 / 0.77 / +4.8%
clickbench_q18/datafusion:vortex-file-compressed 497845492 / 890268564 / -44.1% 🟢 583040672 / 993300116 / -41.3% 🟢 0.85 / 0.90 / -4.7%
clickbench_q19/datafusion:vortex-file-compressed 11696533 / 16409744 / -28.7% 🟢 87035499 / 87616508 / -0.7% 0.13 / 0.19 / -28.2%
clickbench_q20/datafusion:vortex-file-compressed 115133634 / 183409282 / -37.2% 🟢 567553367 / 372398380 / +52.4% 🔴 0.20 / 0.49 / -58.8%
clickbench_q21/datafusion:vortex-file-compressed 160967418 / 264312768 / -39.1% 🟢 693261632 / 466559947 / +48.6% 🔴 0.23 / 0.57 / -59.0%
clickbench_q22/datafusion:vortex-file-compressed 205814474 / 321300689 / -35.9% 🟢 863642947 / 602680479 / +43.3% 🔴 0.24 / 0.53 / -55.3%
clickbench_q23/datafusion:vortex-file-compressed 421704956 / 583115736 / -27.7% 🟢 1209822135 / 858253303 / +41.0% 🔴 0.35 / 0.68 / -48.7%
clickbench_q24/datafusion:vortex-file-compressed 27279599 / 41024666 / -33.5% 🟢 107418440 / 117635547 / -8.7% 0.25 / 0.35 / -27.2%
clickbench_q25/datafusion:vortex-file-compressed 36686613 / 54108476 / -32.2% 🟢 85716992 / 122560192 / -30.1% 🟢 0.43 / 0.44 / -3.1%
clickbench_q26/datafusion:vortex-file-compressed 27839039 / 41402312 / -32.8% 🟢 111147314 / 124965719 / -11.1% 🟢 0.25 / 0.33 / -24.4%
clickbench_q27/datafusion:vortex-file-compressed 162690911 / 271378806 / -40.1% 🟢 609219568 / 451399373 / +35.0% 🔴 0.27 / 0.60 / -55.6%
clickbench_q28/datafusion:vortex-file-compressed 901556416 / 1425963002 / -36.8% 🟢 1032740117 / 1506893167 / -31.5% 🟢 0.87 / 0.95 / -7.7%
clickbench_q29/datafusion:vortex-file-compressed 31843277 / 38018090 / -16.2% 🟢 72498681 / 93477899 / -22.4% 🟢 0.44 / 0.41 / +8.0%
clickbench_q30/datafusion:vortex-file-compressed 120914626 / 171102150 / -29.3% 🟢 172589358 / 249801381 / -30.9% 🟢 0.70 / 0.68 / +2.3%
clickbench_q31/datafusion:vortex-file-compressed 116650400 / 195101996 / -40.2% 🟢 294821462 / 320135302 / -7.9% 0.40 / 0.61 / -35.1%
clickbench_q32/datafusion:vortex-file-compressed 461902089 / 827491971 / -44.2% 🟢 593150446 / 944363975 / -37.2% 🟢 0.78 / 0.88 / -11.1%
clickbench_q33/datafusion:vortex-file-compressed 567317102 / 1008411260 / -43.7% 🟢 838471257 / 1212630290 / -30.9% 🟢 0.68 / 0.83 / -18.6%
clickbench_q34/datafusion:vortex-file-compressed 558788919 / 1008888868 / -44.6% 🟢 837303622 / 1181763217 / -29.1% 🟢 0.67 / 0.85 / -21.8%
clickbench_q35/datafusion:vortex-file-compressed 95485252 / 150893917 / -36.7% 🟢 138835029 / 219337418 / -36.7% 🟢 0.69 / 0.69 / -0.0%
clickbench_q36/datafusion:vortex-file-compressed 43790481 / 57273442 / -23.5% 🟢 89155014 / 117796463 / -24.3% 🟢 0.49 / 0.49 / +1.0%
clickbench_q37/datafusion:vortex-file-compressed 24510734 / 28891052 / -15.2% 🟢 64685466 / 82252628 / -21.4% 🟢 0.38 / 0.35 / +7.9%
clickbench_q38/datafusion:vortex-file-compressed 11752946 / 14946248 / -21.4% 🟢 57843689 / 70249370 / -17.7% 🟢 0.20 / 0.21 / -4.5%
clickbench_q39/datafusion:vortex-file-compressed 93231322 / 111502420 / -16.4% 🟢 144024569 / 183728266 / -21.6% 🟢 0.65 / 0.61 / +6.7%
clickbench_q40/datafusion:vortex-file-compressed 11168318 / 13498886 / -17.3% 🟢 53166477 / 65255804 / -18.5% 🟢 0.21 / 0.21 / +1.5%
clickbench_q41/datafusion:vortex-file-compressed 10351480 / 12773094 / -19.0% 🟢 51140666 / 63842765 / -19.9% 🟢 0.20 / 0.20 / +1.2%
clickbench_q42/datafusion:vortex-file-compressed 12951816 / 13531858 / -4.3% 52411460 / 61497726 / -14.8% 🟢 0.25 / 0.22 / +12.3%
datafusion / vortex-compact / ns (0.629x ✅, 42↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
clickbench_q00/datafusion:vortex-compact 1529016 / 1907006 / -19.8% 🟢 30742496 / 39572904 / -22.3% 🟢 0.05 / 0.05 / +3.2%
clickbench_q01/datafusion:vortex-compact 10769392 / 14530223 / -25.9% 🟢 43595878 / 61056868 / -28.6% 🟢 0.25 / 0.24 / +3.8%
clickbench_q02/datafusion:vortex-compact 34408422 / 56273096 / -38.9% 🟢 65908065 / 99351264 / -33.7% 🟢 0.52 / 0.57 / -7.8%
clickbench_q03/datafusion:vortex-compact 22320113 / 34896204 / -36.0% 🟢 75462668 / 83277741 / -9.4% 0.30 / 0.42 / -29.4%
clickbench_q04/datafusion:vortex-compact 96840861 / 159929766 / -39.4% 🟢 135523446 / 232055944 / -41.6% 🟢 0.71 / 0.69 / +3.7%
clickbench_q05/datafusion:vortex-compact 122584146 / 233630559 / -47.5% 🟢 170722127 / 309869577 / -44.9% 🟢 0.72 / 0.75 / -4.8%
clickbench_q06/datafusion:vortex-compact 1545434 / 1895645 / -18.5% 🟢 31768964 / 37944815 / -16.3% 🟢 0.05 / 0.05 / -2.6%
clickbench_q07/datafusion:vortex-compact 16686370 / 25033066 / -33.3% 🟢 50328054 / 69506541 / -27.6% 🟢 0.33 / 0.36 / -7.9%
clickbench_q08/datafusion:vortex-compact 140821863 / 247871364 / -43.2% 🟢 189755559 / 323208080 / -41.3% 🟢 0.74 / 0.77 / -3.2%
clickbench_q09/datafusion:vortex-compact 273009994 / 393792217 / -30.7% 🟢 319908240 / 455507495 / -29.8% 🟢 0.85 / 0.86 / -1.3%
clickbench_q10/datafusion:vortex-compact 68256328 / 117226104 / -41.8% 🟢 106632624 / 169010200 / -36.9% 🟢 0.64 / 0.69 / -7.7%
clickbench_q11/datafusion:vortex-compact 90268433 / 158418759 / -43.0% 🟢 126242145 / 208415947 / -39.4% 🟢 0.72 / 0.76 / -5.9%
clickbench_q12/datafusion:vortex-compact 121400051 / 226897817 / -46.5% 🟢 177019825 / 310273144 / -42.9% 🟢 0.69 / 0.73 / -6.2%
clickbench_q13/datafusion:vortex-compact 215466081 / 380791222 / -43.4% 🟢 267538531 / 469289007 / -43.0% 🟢 0.81 / 0.81 / -0.7%
clickbench_q14/datafusion:vortex-compact 137972395 / 227172694 / -39.3% 🟢 183243708 / 316245552 / -42.1% 🟢 0.75 / 0.72 / +4.8%
clickbench_q15/datafusion:vortex-compact 109797650 / 189596274 / -42.1% 🟢 151624416 / 257857338 / -41.2% 🟢 0.72 / 0.74 / -1.5%
clickbench_q16/datafusion:vortex-compact 277696178 / 501624896 / -44.6% 🟢 331987915 / 576553452 / -42.4% 🟢 0.84 / 0.87 / -3.9%
clickbench_q17/datafusion:vortex-compact 276350238 / 498578434 / -44.6% 🟢 329636423 / 583603089 / -43.5% 🟢 0.84 / 0.85 / -1.9%
clickbench_q18/datafusion:vortex-compact 511206712 / 890211436 / -42.6% 🟢 569817970 / 988159180 / -42.3% 🟢 0.90 / 0.90 / -0.4%
clickbench_q19/datafusion:vortex-compact 15948865 / 23666205 / -32.6% 🟢 68801134 / 74054377 / -7.1% 0.23 / 0.32 / -27.5%
clickbench_q20/datafusion:vortex-compact 120095966 / 224318381 / -46.5% 🟢 469971033 / 353108726 / +33.1% 🔴 0.26 / 0.64 / -59.8%
clickbench_q21/datafusion:vortex-compact 201175952 / 378116760 / -46.8% 🟢 564582437 / 504016821 / +12.0% 🔴 0.36 / 0.75 / -52.5%
clickbench_q22/datafusion:vortex-compact 318385473 / 606246524 / -47.5% 🟢 652927964 / 719500657 / -9.3% 0.49 / 0.84 / -42.1%
clickbench_q23/datafusion:vortex-compact 991940073 / 1703889878 / -41.8% 🟢 1240287366 / 2085449436 / -40.5% 🟢 0.80 / 0.82 / -2.1%
clickbench_q24/datafusion:vortex-compact 43432872 / 74026157 / -41.3% 🟢 96872706 / 129748697 / -25.3% 🟢 0.45 / 0.57 / -21.4%
clickbench_q25/datafusion:vortex-compact 60020334 / 104736548 / -42.7% 🟢 99062622 / 163276859 / -39.3% 🟢 0.61 / 0.64 / -5.5%
clickbench_q26/datafusion:vortex-compact 43095700 / 73717714 / -41.5% 🟢 95219824 / 132354766 / -28.1% 🟢 0.45 / 0.56 / -18.7%
clickbench_q27/datafusion:vortex-compact 169069858 / 319946218 / -47.2% 🟢 511961459 / 468501261 / +9.3% 0.33 / 0.68 / -51.6%
clickbench_q28/datafusion:vortex-compact 928362715 / 1531265486 / -39.4% 🟢 1094136048 / 1584624375 / -31.0% 🟢 0.85 / 0.97 / -12.2%
clickbench_q29/datafusion:vortex-compact 44878381 / 64454266 / -30.4% 🟢 78039362 / 112972170 / -30.9% 🟢 0.58 / 0.57 / +0.8%
clickbench_q30/datafusion:vortex-compact 150785649 / 262237904 / -42.5% 🟢 199975012 / 328148594 / -39.1% 🟢 0.75 / 0.80 / -5.6%
clickbench_q31/datafusion:vortex-compact 158037358 / 279795554 / -43.5% 🟢 282019190 / 377344731 / -25.3% 🟢 0.56 / 0.74 / -24.4%
clickbench_q32/datafusion:vortex-compact 462158910 / 828113358 / -44.2% 🟢 574345676 / 935373767 / -38.6% 🟢 0.80 / 0.89 / -9.1%
clickbench_q33/datafusion:vortex-compact 580185610 / 1022439194 / -43.3% 🟢 772100317 / 1202163117 / -35.8% 🟢 0.75 / 0.85 / -11.6%
clickbench_q34/datafusion:vortex-compact 565762446 / 1018466426 / -44.4% 🟢 765152454 / 1228322404 / -37.7% 🟢 0.74 / 0.83 / -10.8%
clickbench_q35/datafusion:vortex-compact 104776386 / 161896408 / -35.3% 🟢 142375713 / 230840420 / -38.3% 🟢 0.74 / 0.70 / +4.9%
clickbench_q36/datafusion:vortex-compact 43698124 / 57748602 / -24.3% 🟢 86345651 / 117247989 / -26.4% 🟢 0.51 / 0.49 / +2.8%
clickbench_q37/datafusion:vortex-compact 24770048 / 29625656 / -16.4% 🟢 60556800 / 84230654 / -28.1% 🟢 0.41 / 0.35 / +16.3%
clickbench_q38/datafusion:vortex-compact 12676273 / 16963668 / -25.3% 🟢 54762217 / 73201255 / -25.2% 🟢 0.23 / 0.23 / -0.1%
clickbench_q39/datafusion:vortex-compact 93787968 / 114831748 / -18.3% 🟢 139777452 / 183922140 / -24.0% 🟢 0.67 / 0.62 / +7.5%
clickbench_q40/datafusion:vortex-compact 12663626 / 15583308 / -18.7% 🟢 48556779 / 65289829 / -25.6% 🟢 0.26 / 0.24 / +9.3%
clickbench_q41/datafusion:vortex-compact 13976050 / 17215112 / -18.8% 🟢 48416273 / 64010262 / -24.4% 🟢 0.29 / 0.27 / +7.3%
clickbench_q42/datafusion:vortex-compact 13098380 / 14051164 / -6.8% 48097574 / 59911595 / -19.7% 🟢 0.27 / 0.23 / +16.1%
datafusion / parquet / ns (0.676x ✅, 43↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
clickbench_q00/datafusion:parquet 1561377 / 1919190 / -18.6% 🟢 66794792 / 74850592 / -10.8% 🟢 0.02 / 0.03 / -8.8%
clickbench_q01/datafusion:parquet 13491017 / 16752490 / -19.5% 🟢 87771974 / 110526433 / -20.6% 🟢 0.15 / 0.15 / +1.4%
clickbench_q02/datafusion:parquet 25062870 / 31948106 / -21.6% 🟢 105497637 / 137411338 / -23.2% 🟢 0.24 / 0.23 / +2.2%
clickbench_q03/datafusion:parquet 21213158 / 27828592 / -23.8% 🟢 111465412 / 135624270 / -17.8% 🟢 0.19 / 0.21 / -7.3%
clickbench_q04/datafusion:parquet 100925380 / 173589226 / -41.9% 🟢 182383824 / 299970538 / -39.2% 🟢 0.55 / 0.58 / -4.4%
clickbench_q05/datafusion:parquet 136879472 / 234298674 / -41.6% 🟢 225655191 / 369069805 / -38.9% 🟢 0.61 / 0.63 / -4.4%
clickbench_q06/datafusion:parquet 1578432 / 1920754 / -17.8% 🟢 66137569 / 75838884 / -12.8% 🟢 0.02 / 0.03 / -5.8%
clickbench_q07/datafusion:parquet 16064939 / 18185464 / -11.7% 🟢 87785079 / 108352263 / -19.0% 🟢 0.18 / 0.17 / +9.0%
clickbench_q08/datafusion:parquet 134889288 / 232312739 / -41.9% 🟢 239403351 / 373495391 / -35.9% 🟢 0.56 / 0.62 / -9.4%
clickbench_q09/datafusion:parquet 258350943 / 359187688 / -28.1% 🟢 371299222 / 499471247 / -25.7% 🟢 0.70 / 0.72 / -3.2%
clickbench_q10/datafusion:parquet 45344986 / 71999924 / -37.0% 🟢 134997570 / 192105415 / -29.7% 🟢 0.34 / 0.37 / -10.4%
clickbench_q11/datafusion:parquet 54445742 / 87272802 / -37.6% 🟢 144938797 / 208633673 / -30.5% 🟢 0.38 / 0.42 / -10.2%
clickbench_q12/datafusion:parquet 146254584 / 238996602 / -38.8% 🟢 238958675 / 374133171 / -36.1% 🟢 0.61 / 0.64 / -4.2%
clickbench_q13/datafusion:parquet 223046580 / 397056362 / -43.8% 🟢 322433848 / 539930731 / -40.3% 🟢 0.69 / 0.74 / -5.9%
clickbench_q14/datafusion:parquet 156415427 / 255107037 / -38.7% 🟢 256130900 / 399168417 / -35.8% 🟢 0.61 / 0.64 / -4.4%
clickbench_q15/datafusion:parquet 112638940 / 196985632 / -42.8% 🟢 198519960 / 324885504 / -38.9% 🟢 0.57 / 0.61 / -6.4%
clickbench_q16/datafusion:parquet 260194126 / 476857338 / -45.4% 🟢 360074021 / 625755324 / -42.5% 🟢 0.72 / 0.76 / -5.2%
clickbench_q17/datafusion:parquet 260191804 / 465933312 / -44.2% 🟢 359019994 / 628220577 / -42.9% 🟢 0.72 / 0.74 / -2.3%
clickbench_q18/datafusion:parquet 545443698 / 892808319 / -38.9% 🟢 628779142 / 1063188462 / -40.9% 🟢 0.87 / 0.84 / +3.3%
clickbench_q19/datafusion:parquet 18353906 / 23982830 / -23.5% 🟢 99112661 / 118401396 / -16.3% 🟢 0.19 / 0.20 / -8.6%
clickbench_q20/datafusion:parquet 289293624 / 453125058 / -36.2% 🟢 591261752 / 607849669 / -2.7% 0.49 / 0.75 / -34.4%
clickbench_q21/datafusion:parquet 271156668 / 434209160 / -37.6% 🟢 619371893 / 617457088 / +0.3% 0.44 / 0.70 / -37.7%
clickbench_q22/datafusion:parquet 387978170 / 609464877 / -36.3% 🟢 925647485 / 831305469 / +11.3% 🔴 0.42 / 0.73 / -42.8%
clickbench_q23/datafusion:parquet 1426714324 / 2397198856 / -40.5% 🟢 2023305534 / 2898438302 / -30.2% 🟢 0.71 / 0.83 / -14.7%
clickbench_q24/datafusion:parquet 31553324 / 40765839 / -22.6% 🟢 122994691 / 140703816 / -12.6% 🟢 0.26 / 0.29 / -11.5%
clickbench_q25/datafusion:parquet 52476120 / 87921712 / -40.3% 🟢 150616376 / 197015927 / -23.6% 🟢 0.35 / 0.45 / -21.9%
clickbench_q26/datafusion:parquet 32321974 / 42415082 / -23.8% 🟢 123998925 / 144479578 / -14.2% 🟢 0.26 / 0.29 / -11.2%
clickbench_q27/datafusion:parquet 320653238 / 526143124 / -39.1% 🟢 581150972 / 673606984 / -13.7% 🟢 0.55 / 0.78 / -29.4%
clickbench_q28/datafusion:parquet 957281200 / 1532893528 / -37.6% 🟢 1086583268 / 1648480377 / -34.1% 🟢 0.88 / 0.93 / -5.3%
clickbench_q29/datafusion:parquet 33894404 / 40404672 / -16.1% 🟢 112506701 / 144329488 / -22.0% 🟢 0.30 / 0.28 / +7.6%
clickbench_q30/datafusion:parquet 132216635 / 232152649 / -43.0% 🟢 266487344 / 385895129 / -30.9% 🟢 0.50 / 0.60 / -17.5%
clickbench_q31/datafusion:parquet 149367366 / 263763431 / -43.4% 🟢 324425722 / 432008273 / -24.9% 🟢 0.46 / 0.61 / -24.6%
clickbench_q32/datafusion:parquet 475694616 / 844309537 / -43.7% 🟢 590144244 / 1032339393 / -42.8% 🟢 0.81 / 0.82 / -1.4%
clickbench_q33/datafusion:parquet 680438016 / 1087611999 / -37.4% 🟢 800882190 / 1252587388 / -36.1% 🟢 0.85 / 0.87 / -2.2%
clickbench_q34/datafusion:parquet 664509268 / 1090460556 / -39.1% 🟢 815494873 / 1292142935 / -36.9% 🟢 0.81 / 0.84 / -3.4%
clickbench_q35/datafusion:parquet 102365671 / 163360808 / -37.3% 🟢 182185828 / 283058452 / -35.6% 🟢 0.56 / 0.58 / -2.6%
clickbench_q36/datafusion:parquet 85011672 / 107740537 / -21.1% 🟢 166282039 / 216068578 / -23.0% 🟢 0.51 / 0.50 / +2.5%
clickbench_q37/datafusion:parquet 40663515 / 48355936 / -15.9% 🟢 114462089 / 143962033 / -20.5% 🟢 0.36 / 0.34 / +5.8%
clickbench_q38/datafusion:parquet 53742176 / 65593433 / -18.1% 🟢 131327345 / 167786231 / -21.7% 🟢 0.41 / 0.39 / +4.7%
clickbench_q39/datafusion:parquet 169566750 / 202941156 / -16.4% 🟢 253229830 / 321776316 / -21.3% 🟢 0.67 / 0.63 / +6.2%
clickbench_q40/datafusion:parquet 21377862 / 26958328 / -20.7% 🟢 96711034 / 124836837 / -22.5% 🟢 0.22 / 0.22 / +2.4%
clickbench_q41/datafusion:parquet 20230418 / 25953585 / -22.1% 🟢 96256531 / 118085541 / -18.5% 🟢 0.21 / 0.22 / -4.4%
clickbench_q42/datafusion:parquet 22974496 / 25981906 / -11.6% 🟢 96388866 / 117291859 / -17.8% 🟢 0.24 / 0.22 / +7.6%
duckdb / vortex-file-compressed / ns (0.712x ✅, 41↑ 1↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
clickbench_q00/duckdb:vortex-file-compressed 2296348 / 2027081 / +13.3% 🔴 27398862 / 74182868 / -63.1% 🟢 0.08 / 0.03 / +206.7%
clickbench_q01/duckdb:vortex-file-compressed 14304020 / 18922216 / -24.4% 🟢 31496214 / 102934726 / -69.4% 🟢 0.45 / 0.18 / +147.1%
clickbench_q02/duckdb:vortex-file-compressed 17986550 / 24642278 / -27.0% 🟢 42123036 / 109768357 / -61.6% 🟢 0.43 / 0.22 / +90.2%
clickbench_q03/duckdb:vortex-file-compressed 23904861 / 33043206 / -27.7% 🟢 82616201 / 113511305 / -27.2% 🟢 0.29 / 0.29 / -0.6%
clickbench_q04/duckdb:vortex-file-compressed 99357667 / 132086008 / -24.8% 🟢 140013880 / 206484585 / -32.2% 🟢 0.71 / 0.64 / +10.9%
clickbench_q05/duckdb:vortex-file-compressed 99087917 / 156892894 / -36.8% 🟢 123896340 / 242855677 / -49.0% 🟢 0.80 / 0.65 / +23.8%
clickbench_q06/duckdb:vortex-file-compressed 2713376 / 2475513 / +9.6% 33881446 / 93217768 / -63.7% 🟢 0.08 / 0.03 / +201.6%
clickbench_q07/duckdb:vortex-file-compressed 17764950 / 22566738 / -21.3% 🟢 35442245 / 99078923 / -64.2% 🟢 0.50 / 0.23 / +120.1%
clickbench_q08/duckdb:vortex-file-compressed 127929682 / 171690977 / -25.5% 🟢 181899078 / 288712675 / -37.0% 🟢 0.70 / 0.59 / +18.3%
clickbench_q09/duckdb:vortex-file-compressed 164426670 / 211619386 / -22.3% 🟢 218447337 / 335311698 / -34.9% 🟢 0.75 / 0.63 / +19.3%
clickbench_q10/duckdb:vortex-file-compressed 57295608 / 73943003 / -22.5% 🟢 111621753 / 182403735 / -38.8% 🟢 0.51 / 0.41 / +26.6%
clickbench_q11/duckdb:vortex-file-compressed 75414301 / 92282208 / -18.3% 🟢 124103489 / 197401273 / -37.1% 🟢 0.61 / 0.47 / +30.0%
clickbench_q12/duckdb:vortex-file-compressed 112773300 / 172712882 / -34.7% 🟢 143232576 / 291647616 / -50.9% 🟢 0.79 / 0.59 / +33.0%
clickbench_q13/duckdb:vortex-file-compressed 214722798 / 319893279 / -32.9% 🟢 280583819 / 446116731 / -37.1% 🟢 0.77 / 0.72 / +6.7%
clickbench_q14/duckdb:vortex-file-compressed 130528148 / 188509591 / -30.8% 🟢 163914867 / 311323839 / -47.3% 🟢 0.80 / 0.61 / +31.5%
clickbench_q15/duckdb:vortex-file-compressed 129705412 / 190257292 / -31.8% 🟢 174590250 / 289953909 / -39.8% 🟢 0.74 / 0.66 / +13.2%
clickbench_q16/duckdb:vortex-file-compressed 251484096 / 375892397 / -33.1% 🟢 301676193 / 484219247 / -37.7% 🟢 0.83 / 0.78 / +7.4%
clickbench_q17/duckdb:vortex-file-compressed 256915669 / 366346696 / -29.9% 🟢 299846898 / 490955044 / -38.9% 🟢 0.86 / 0.75 / +14.8%
clickbench_q18/duckdb:vortex-file-compressed 475892512 / 735897868 / -35.3% 🟢 526233666 / 897007083 / -41.3% 🟢 0.90 / 0.82 / +10.2%
clickbench_q19/duckdb:vortex-file-compressed 18103320 / 26901372 / -32.7% 🟢 70956196 / 110019419 / -35.5% 🟢 0.26 / 0.24 / +4.3%
clickbench_q20/duckdb:vortex-file-compressed 158822570 / 243208596 / -34.7% 🟢 389271494 / 450659328 / -13.6% 🟢 0.41 / 0.54 / -24.4%
clickbench_q21/duckdb:vortex-file-compressed 226352858 / 365200389 / -38.0% 🟢 703981148 / 658540060 / +6.9% 0.32 / 0.55 / -42.0%
clickbench_q22/duckdb:vortex-file-compressed 395619872 / 561729649 / -29.6% 🟢 986366009 / 969460395 / +1.7% 0.40 / 0.58 / -30.8%
clickbench_q23/duckdb:vortex-file-compressed 68746090 / 123060230 / -44.1% 🟢 169559306 / 233902433 / -27.5% 🟢 0.41 / 0.53 / -22.9%
clickbench_q24/duckdb:vortex-file-compressed 29180110 / 38832162 / -24.9% 🟢 49849282 / 98636241 / -49.5% 🟢 0.59 / 0.39 / +48.7%
clickbench_q25/duckdb:vortex-file-compressed 43805710 / 60958140 / -28.1% 🟢 70264200 / 145485553 / -51.7% 🟢 0.62 / 0.42 / +48.8%
clickbench_q26/duckdb:vortex-file-compressed 25080684 / 34840538 / -28.0% 🟢 47314403 / 85977098 / -45.0% 🟢 0.53 / 0.41 / +30.8%
clickbench_q27/duckdb:vortex-file-compressed 178238360 / 247733218 / -28.1% 🟢 623232999 / 497374838 / +25.3% 🔴 0.29 / 0.50 / -42.6%
clickbench_q28/duckdb:vortex-file-compressed 973423086 / 1179517654 / -17.5% 🟢 1084140394 / 1556108099 / -30.3% 🟢 0.90 / 0.76 / +18.5%
clickbench_q29/duckdb:vortex-file-compressed 23097582 / 30237942 / -23.6% 🟢 48474476 / 117412150 / -58.7% 🟢 0.48 / 0.26 / +85.0%
clickbench_q30/duckdb:vortex-file-compressed 120099589 / 167712246 / -28.4% 🟢 184693951 / 296147810 / -37.6% 🟢 0.65 / 0.57 / +14.8%
clickbench_q31/duckdb:vortex-file-compressed 160319311 / 312298199 / -48.7% 🟢 326623809 / 479350080 / -31.9% 🟢 0.49 / 0.65 / -24.7%
clickbench_q32/duckdb:vortex-file-compressed 588200190 / 910016020 / -35.4% 🟢 651819349 / 1067928290 / -39.0% 🟢 0.90 / 0.85 / +5.9%
clickbench_q33/duckdb:vortex-file-compressed 582133552 / 1047606028 / -44.4% 🟢 865516778 / 1265611491 / -31.6% 🟢 0.67 / 0.83 / -18.7%
clickbench_q34/duckdb:vortex-file-compressed 625956908 / 1075733080 / -41.8% 🟢 857063008 / 1343204094 / -36.2% 🟢 0.73 / 0.80 / -8.8%
clickbench_q35/duckdb:vortex-file-compressed 149627000 / 209806304 / -28.7% 🟢 182166097 / 288696916 / -36.9% 🟢 0.82 / 0.73 / +13.0%
clickbench_q36/duckdb:vortex-file-compressed 19481206 / 30321250 / -35.8% 🟢 38719303 / 127046859 / -69.5% 🟢 0.50 / 0.24 / +110.8%
clickbench_q37/duckdb:vortex-file-compressed 14181286 / 18585794 / -23.7% 🟢 28102694 / 97414496 / -71.2% 🟢 0.50 / 0.19 / +164.5%
clickbench_q38/duckdb:vortex-file-compressed 15111668 / 22151942 / -31.8% 🟢 41613938 / 88265282 / -52.9% 🟢 0.36 / 0.25 / +44.7%
clickbench_q39/duckdb:vortex-file-compressed 31230918 / 45708143 / -31.7% 🟢 54386726 / 126094748 / -56.9% 🟢 0.57 / 0.36 / +58.4%
clickbench_q40/duckdb:vortex-file-compressed 12767700 / 17015213 / -25.0% 🟢 29755119 / 72918517 / -59.2% 🟢 0.43 / 0.23 / +83.9%
clickbench_q41/duckdb:vortex-file-compressed 14064060 / 19834565 / -29.1% 🟢 32607711 / 92012646 / -64.6% 🟢 0.43 / 0.22 / +100.1%
clickbench_q42/duckdb:vortex-file-compressed 14596530 / 17801492 / -18.0% 🟢 31018226 / 100369932 / -69.1% 🟢 0.47 / 0.18 / +165.3%
duckdb / vortex-compact / ns (0.752x ✅, 38↑ 1↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
clickbench_q00/duckdb:vortex-compact 2313094 / 2031874 / +13.8% 🔴 26609163 / 76049731 / -65.0% 🟢 0.09 / 0.03 / +225.4%
clickbench_q01/duckdb:vortex-compact 15245820 / 20816995 / -26.8% 🟢 30932512 / 99546746 / -68.9% 🟢 0.49 / 0.21 / +135.7%
clickbench_q02/duckdb:vortex-compact 32794894 / 35148480 / -6.7% 51046665 / 118000777 / -56.7% 🟢 0.64 / 0.30 / +115.7%
clickbench_q03/duckdb:vortex-compact 29821278 / 36217868 / -17.7% 🟢 65824308 / 96547933 / -31.8% 🟢 0.45 / 0.38 / +20.8%
clickbench_q04/duckdb:vortex-compact 106277801 / 140655722 / -24.4% 🟢 134311175 / 204611014 / -34.4% 🟢 0.79 / 0.69 / +15.1%
clickbench_q05/duckdb:vortex-compact 116171934 / 161467103 / -28.1% 🟢 137339543 / 240171978 / -42.8% 🟢 0.85 / 0.67 / +25.8%
clickbench_q06/duckdb:vortex-compact 2638132 / 2436110 / +8.3% 32425984 / 98218113 / -67.0% 🟢 0.08 / 0.02 / +228.0%
clickbench_q07/duckdb:vortex-compact 19048994 / 22330820 / -14.7% 🟢 35273121 / 103007988 / -65.8% 🟢 0.54 / 0.22 / +149.1%
clickbench_q08/duckdb:vortex-compact 145470153 / 188949864 / -23.0% 🟢 181570417 / 296039617 / -38.7% 🟢 0.80 / 0.64 / +25.5%
clickbench_q09/duckdb:vortex-compact 214311638 / 264714688 / -19.0% 🟢 245606011 / 380762899 / -35.5% 🟢 0.87 / 0.70 / +25.5%
clickbench_q10/duckdb:vortex-compact 80909591 / 87139406 / -7.1% 118971059 / 193233512 / -38.4% 🟢 0.68 / 0.45 / +50.8%
clickbench_q11/duckdb:vortex-compact 103352989 / 109874072 / -5.9% 139179772 / 221608971 / -37.2% 🟢 0.74 / 0.50 / +49.8%
clickbench_q12/duckdb:vortex-compact 138645742 / 188918949 / -26.6% 🟢 170098821 / 295149355 / -42.4% 🟢 0.82 / 0.64 / +27.3%
clickbench_q13/duckdb:vortex-compact 253469584 / 347262398 / -27.0% 🟢 300642284 / 471725699 / -36.3% 🟢 0.84 / 0.74 / +14.5%
clickbench_q14/duckdb:vortex-compact 156687544 / 204988294 / -23.6% 🟢 188963320 / 309467960 / -38.9% 🟢 0.83 / 0.66 / +25.2%
clickbench_q15/duckdb:vortex-compact 132989722 / 190915928 / -30.3% 🟢 168144763 / 296349946 / -43.3% 🟢 0.79 / 0.64 / +22.8%
clickbench_q16/duckdb:vortex-compact 282604424 / 396464476 / -28.7% 🟢 320702902 / 512060370 / -37.4% 🟢 0.88 / 0.77 / +13.8%
clickbench_q17/duckdb:vortex-compact 286221039 / 390814082 / -26.8% 🟢 315653924 / 494725454 / -36.2% 🟢 0.91 / 0.79 / +14.8%
clickbench_q18/duckdb:vortex-compact 490892544 / 735545736 / -33.3% 🟢 549275816 / 844127190 / -34.9% 🟢 0.89 / 0.87 / +2.6%
clickbench_q19/duckdb:vortex-compact 20849772 / 26120072 / -20.2% 🟢 54209574 / 101060559 / -46.4% 🟢 0.38 / 0.26 / +48.8%
clickbench_q20/duckdb:vortex-compact 156713291 / 215531689 / -27.3% 🟢 311843016 / 393942166 / -20.8% 🟢 0.50 / 0.55 / -8.1%
clickbench_q21/duckdb:vortex-compact 267060894 / 357406499 / -25.3% 🟢 640724076 / 659382583 / -2.8% 0.42 / 0.54 / -23.1%
clickbench_q22/duckdb:vortex-compact 565565201 / 685398222 / -17.5% 🟢 985723825 / 1083407985 / -9.0% 0.57 / 0.63 / -9.3%
clickbench_q23/duckdb:vortex-compact 105434990 / 199956518 / -47.3% 🟢 174768698 / 275181591 / -36.5% 🟢 0.60 / 0.73 / -17.0%
clickbench_q24/duckdb:vortex-compact 34893293 / 45820666 / -23.8% 🟢 49009815 / 101356981 / -51.6% 🟢 0.71 / 0.45 / +57.5%
clickbench_q25/duckdb:vortex-compact 72967552 / 82286288 / -11.3% 🟢 95887914 / 148208445 / -35.3% 🟢 0.76 / 0.56 / +37.1%
clickbench_q26/duckdb:vortex-compact 28119382 / 40105709 / -29.9% 🟢 48014505 / 92278667 / -48.0% 🟢 0.59 / 0.43 / +34.7%
clickbench_q27/duckdb:vortex-compact 290852602 / 398412841 / -27.0% 🟢 598318036 / 648638006 / -7.8% 0.49 / 0.61 / -20.9%
clickbench_q28/duckdb:vortex-compact 1002732611 / 1132281209 / -11.4% 🟢 1131404427 / 1368371737 / -17.3% 🟢 0.89 / 0.83 / +7.1%
clickbench_q29/duckdb:vortex-compact 34936049 / 42210434 / -17.2% 🟢 57282376 / 126108818 / -54.6% 🟢 0.61 / 0.33 / +82.2%
clickbench_q30/duckdb:vortex-compact 173622832 / 206369386 / -15.9% 🟢 215792839 / 315125270 / -31.5% 🟢 0.80 / 0.65 / +22.9%
clickbench_q31/duckdb:vortex-compact 223914614 / 333340703 / -32.8% 🟢 347168531 / 543860652 / -36.2% 🟢 0.64 / 0.61 / +5.2%
clickbench_q32/duckdb:vortex-compact 593340958 / 942690998 / -37.1% 🟢 639722091 / 1083279379 / -40.9% 🟢 0.93 / 0.87 / +6.6%
clickbench_q33/duckdb:vortex-compact 575979716 / 982286930 / -41.4% 🟢 797697457 / 1208561522 / -34.0% 🟢 0.72 / 0.81 / -11.2%
clickbench_q34/duckdb:vortex-compact 627820819 / 1027511486 / -38.9% 🟢 842220986 / 1302578657 / -35.3% 🟢 0.75 / 0.79 / -5.5%
clickbench_q35/duckdb:vortex-compact 155886100 / 220235765 / -29.2% 🟢 186435880 / 309804041 / -39.8% 🟢 0.84 / 0.71 / +17.6%
clickbench_q36/duckdb:vortex-compact 18350072 / 29525859 / -37.9% 🟢 34626613 / 132127189 / -73.8% 🟢 0.53 / 0.22 / +137.1%
clickbench_q37/duckdb:vortex-compact 16431244 / 22755177 / -27.8% 🟢 28506008 / 111251866 / -74.4% 🟢 0.58 / 0.20 / +181.8%
clickbench_q38/duckdb:vortex-compact 14355392 / 24738896 / -42.0% 🟢 30424567 / 114339673 / -73.4% 🟢 0.47 / 0.22 / +118.1%
clickbench_q39/duckdb:vortex-compact 32636429 / 50319936 / -35.1% 🟢 57976405 / 134453267 / -56.9% 🟢 0.56 / 0.37 / +50.4%
clickbench_q40/duckdb:vortex-compact 15056782 / 20440657 / -26.3% 🟢 30148492 / 82500576 / -63.5% 🟢 0.50 / 0.25 / +101.6%
clickbench_q41/duckdb:vortex-compact 15595498 / 22595871 / -31.0% 🟢 29799245 / 91147742 / -67.3% 🟢 0.52 / 0.25 / +111.1%
clickbench_q42/duckdb:vortex-compact 13557693 / 18301276 / -25.9% 🟢 29198054 / 89509871 / -67.4% 🟢 0.46 / 0.20 / +127.1%
duckdb / parquet / ns (0.745x ✅, 43↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
clickbench_q00/duckdb:parquet 10271454 / 17294224 / -40.6% 🟢 21717202 / 42959747 / -49.4% 🟢 0.47 / 0.40 / +17.5%
clickbench_q01/duckdb:parquet 15062638 / 17116173 / -12.0% 🟢 28591048 / 35449707 / -19.3% 🟢 0.53 / 0.48 / +9.1%
clickbench_q02/duckdb:parquet 23232620 / 26023850 / -10.7% 🟢 60287159 / 56610092 / +6.5% 0.39 / 0.46 / -16.2%
clickbench_q03/duckdb:parquet 21547882 / 28860072 / -25.3% 🟢 96252674 / 82198516 / +17.1% 🔴 0.22 / 0.35 / -36.2%
clickbench_q04/duckdb:parquet 103624734 / 132846530 / -22.0% 🟢 164260671 / 180005450 / -8.7% 0.63 / 0.74 / -14.5%
clickbench_q05/duckdb:parquet 123522682 / 185947438 / -33.6% 🟢 218701433 / 251074512 / -12.9% 🟢 0.56 / 0.74 / -23.7%
clickbench_q06/duckdb:parquet 20731686 / 23470941 / -11.7% 🟢 33834954 / 45618259 / -25.8% 🟢 0.61 / 0.51 / +19.1%
clickbench_q07/duckdb:parquet 16112218 / 20496084 / -21.4% 🟢 29340196 / 41693954 / -29.6% 🟢 0.55 / 0.49 / +11.7%
clickbench_q08/duckdb:parquet 127026592 / 165007386 / -23.0% 🟢 248806428 / 246086562 / +1.1% 0.51 / 0.67 / -23.9%
clickbench_q09/duckdb:parquet 175181110 / 225912234 / -22.5% 🟢 325635139 / 302223498 / +7.7% 0.54 / 0.75 / -28.0%
clickbench_q10/duckdb:parquet 47308020 / 66591129 / -29.0% 🟢 113942911 / 113298197 / +0.6% 0.42 / 0.59 / -29.4%
clickbench_q11/duckdb:parquet 54360923 / 68925530 / -21.1% 🟢 127941802 / 124487335 / +2.8% 0.42 / 0.55 / -23.3%
clickbench_q12/duckdb:parquet 135841874 / 174098502 / -22.0% 🟢 226551001 / 289501006 / -21.7% 🟢 0.60 / 0.60 / -0.3%
clickbench_q13/duckdb:parquet 229308384 / 312693498 / -26.7% 🟢 379897331 / 423492076 / -10.3% 🟢 0.60 / 0.74 / -18.3%
clickbench_q14/duckdb:parquet 150882254 / 192515028 / -21.6% 🟢 241969805 / 274545496 / -11.9% 🟢 0.62 / 0.70 / -11.1%
clickbench_q15/duckdb:parquet 134980880 / 187604224 / -28.1% 🟢 170893825 / 222896426 / -23.3% 🟢 0.79 / 0.84 / -6.2%
clickbench_q16/duckdb:parquet 269545476 / 354766567 / -24.0% 🟢 379450295 / 476751942 / -20.4% 🟢 0.71 / 0.74 / -4.5%
clickbench_q17/duckdb:parquet 276919980 / 365338842 / -24.2% 🟢 381132648 / 453245776 / -15.9% 🟢 0.73 / 0.81 / -9.9%
clickbench_q18/duckdb:parquet 480964569 / 692554676 / -30.6% 🟢 634592978 / 889889445 / -28.7% 🟢 0.76 / 0.78 / -2.6%
clickbench_q19/duckdb:parquet 17568955 / 23652302 / -25.7% 🟢 72162110 / 63099384 / +14.4% 🔴 0.24 / 0.37 / -35.0%
clickbench_q20/duckdb:parquet 185794078 / 265328384 / -30.0% 🟢 497172520 / 444996793 / +11.7% 🔴 0.37 / 0.60 / -37.3%
clickbench_q21/duckdb:parquet 236339282 / 328588344 / -28.1% 🟢 669011827 / 572030962 / +17.0% 🔴 0.35 / 0.57 / -38.5%
clickbench_q22/duckdb:parquet 356121031 / 477184879 / -25.4% 🟢 993663618 / 723773855 / +37.3% 🔴 0.36 / 0.66 / -45.6%
clickbench_q23/duckdb:parquet 161941786 / 263031605 / -38.4% 🟢 229898445 / 327716709 / -29.8% 🟢 0.70 / 0.80 / -12.2%
clickbench_q24/duckdb:parquet 51307074 / 79501993 / -35.5% 🟢 79335564 / 112129989 / -29.2% 🟢 0.65 / 0.71 / -8.8%
clickbench_q25/duckdb:parquet 64271928 / 74316556 / -13.5% 🟢 182909186 / 168117217 / +8.8% 0.35 / 0.44 / -20.5%
clickbench_q26/duckdb:parquet 36044405 / 57407063 / -37.2% 🟢 64181520 / 89552877 / -28.3% 🟢 0.56 / 0.64 / -12.4%
clickbench_q27/duckdb:parquet 211134576 / 269327945 / -21.6% 🟢 523023620 / 486073004 / +7.6% 0.40 / 0.55 / -27.1%
clickbench_q28/duckdb:parquet 1542777989 / 1721897053 / -10.4% 🟢 1960393892 / 2234437927 / -12.3% 🟢 0.79 / 0.77 / +2.1%
clickbench_q29/duckdb:parquet 24587393 / 29531124 / -16.7% 🟢 51375483 / 59631145 / -13.8% 🟢 0.48 / 0.50 / -3.4%
clickbench_q30/duckdb:parquet 141689288 / 177233130 / -20.1% 🟢 333773682 / 313183348 / +6.6% 0.42 / 0.57 / -25.0%
clickbench_q31/duckdb:parquet 177565084 / 320584399 / -44.6% 🟢 488808467 / 546116146 / -10.5% 🟢 0.36 / 0.59 / -38.1%
clickbench_q32/duckdb:parquet 592796152 / 928010454 / -36.1% 🟢 694584799 / 1045135561 / -33.5% 🟢 0.85 / 0.89 / -3.9%
clickbench_q33/duckdb:parquet 595116423 / 787106722 / -24.4% 🟢 747112766 / 966898061 / -22.7% 🟢 0.80 / 0.81 / -2.1%
clickbench_q34/duckdb:parquet 591410976 / 800966443 / -26.2% 🟢 721164938 / 995761456 / -27.6% 🟢 0.82 / 0.80 / +2.0%
clickbench_q35/duckdb:parquet 147508399 / 208435098 / -29.2% 🟢 170786943 / 247917729 / -31.1% 🟢 0.86 / 0.84 / +2.7%
clickbench_q36/duckdb:parquet 40079439 / 51416306 / -22.0% 🟢 49759412 / 72139474 / -31.0% 🟢 0.81 / 0.71 / +13.0%
clickbench_q37/duckdb:parquet 28019519 / 37047140 / -24.4% 🟢 37653464 / 56900149 / -33.8% 🟢 0.74 / 0.65 / +14.3%
clickbench_q38/duckdb:parquet 28760788 / 38399276 / -25.1% 🟢 41673558 / 60000297 / -30.5% 🟢 0.69 / 0.64 / +7.8%
clickbench_q39/duckdb:parquet 70492954 / 89664714 / -21.4% 🟢 81571321 / 109822014 / -25.7% 🟢 0.86 / 0.82 / +5.8%
clickbench_q40/duckdb:parquet 16754764 / 21028388 / -20.3% 🟢 24890269 / 40395304 / -38.4% 🟢 0.67 / 0.52 / +29.3%
clickbench_q41/duckdb:parquet 15818526 / 22196811 / -28.7% 🟢 26206446 / 43787306 / -40.2% 🟢 0.60 / 0.51 / +19.1%
clickbench_q42/duckdb:parquet 18282864 / 23527686 / -22.3% 🟢 27584017 / 49262672 / -44.0% 🟢 0.66 / 0.48 / +38.8%
File Size Changes (83 files changed, -0.0% overall, 41↑ 42↓)
File Scale Format Base HEAD Change %
hits_21.vortex 1.0 vortex-file-compressed 92.25 MB 92.38 MB +123.72 KB +0.1%
hits_74.vortex 1.0 vortex-file-compressed 118.88 MB 118.88 MB +2.90 KB +0.0%
hits_71.vortex 1.0 vortex-file-compressed 100.95 MB 100.96 MB +976 B +0.0%
hits_8.vortex 1.0 vortex-file-compressed 91.74 MB 91.74 MB +672 B +0.0%
hits_2.vortex 1.0 vortex-file-compressed 186.82 MB 186.82 MB +1.06 KB +0.0%
hits_14.vortex 1.0 vortex-file-compressed 109.62 MB 109.62 MB +344 B +0.0%
hits_95.vortex 1.0 vortex-file-compressed 95.35 MB 95.35 MB +296 B +0.0%
hits_59.vortex 1.0 vortex-file-compressed 100.65 MB 100.65 MB +312 B +0.0%
hits_88.vortex 1.0 vortex-file-compressed 109.95 MB 109.95 MB +256 B +0.0%
hits_54.vortex 1.0 vortex-file-compressed 221.18 MB 221.18 MB +256 B +0.0%
hits_94.vortex 1.0 vortex-file-compressed 137.84 MB 137.84 MB +128 B +0.0%
hits_15.vortex 1.0 vortex-file-compressed 88.29 MB 88.29 MB +80 B +0.0%
hits_49.vortex 1.0 vortex-file-compressed 74.57 MB 74.57 MB +64 B +0.0%
hits_75.vortex 1.0 vortex-file-compressed 61.68 MB 61.68 MB +48 B +0.0%
hits_37.vortex 1.0 vortex-file-compressed 84.35 MB 84.35 MB +64 B +0.0%
hits_17.vortex 1.0 vortex-file-compressed 86.30 MB 86.30 MB +64 B +0.0%
hits_56.vortex 1.0 vortex-file-compressed 122.41 MB 122.41 MB +80 B +0.0%
hits_66.vortex 1.0 vortex-file-compressed 89.15 MB 89.15 MB +48 B +0.0%
hits_85.vortex 1.0 vortex-file-compressed 90.94 MB 90.94 MB +48 B +0.0%
hits_32.vortex 1.0 vortex-file-compressed 65.46 MB 65.46 MB +32 B +0.0%
hits_44.vortex 1.0 vortex-file-compressed 187.12 MB 187.12 MB +80 B +0.0%
hits_0.vortex 1.0 vortex-file-compressed 87.76 MB 87.76 MB +32 B +0.0%
hits_5.vortex 1.0 vortex-file-compressed 91.45 MB 91.45 MB +32 B +0.0%
hits_7.vortex 1.0 vortex-file-compressed 92.45 MB 92.45 MB +32 B +0.0%
hits_91.vortex 1.0 vortex-file-compressed 95.92 MB 95.92 MB +32 B +0.0%
hits_34.vortex 1.0 vortex-file-compressed 96.61 MB 96.61 MB +32 B +0.0%
hits_77.vortex 1.0 vortex-file-compressed 168.86 MB 168.86 MB +48 B +0.0%
hits_40.vortex 1.0 vortex-file-compressed 116.81 MB 116.81 MB +32 B +0.0%
hits_36.vortex 1.0 vortex-file-compressed 66.37 MB 66.37 MB +16 B +0.0%
hits_19.vortex 1.0 vortex-file-compressed 72.19 MB 72.19 MB +16 B +0.0%
hits_30.vortex 1.0 vortex-file-compressed 85.22 MB 85.22 MB +16 B +0.0%
hits_70.vortex 1.0 vortex-file-compressed 92.23 MB 92.23 MB +16 B +0.0%
hits_52.vortex 1.0 vortex-file-compressed 102.95 MB 102.95 MB +16 B +0.0%
hits_97.vortex 1.0 vortex-file-compressed 106.36 MB 106.36 MB +16 B +0.0%
hits_26.vortex 1.0 vortex-file-compressed 108.66 MB 108.66 MB +16 B +0.0%
hits_41.vortex 1.0 vortex-file-compressed 223.98 MB 223.98 MB +32 B +0.0%
hits_45.vortex 1.0 vortex-file-compressed 121.13 MB 121.13 MB +16 B +0.0%
hits_1.vortex 1.0 vortex-file-compressed 137.87 MB 137.87 MB +16 B +0.0%
hits_78.vortex 1.0 vortex-file-compressed 163.88 MB 163.88 MB +16 B +0.0%
hits_55.vortex 1.0 vortex-file-compressed 168.11 MB 168.11 MB +16 B +0.0%
hits_43.vortex 1.0 vortex-file-compressed 227.63 MB 227.63 MB +16 B +0.0%
hits_79.vortex 1.0 vortex-file-compressed 143.47 MB 143.47 MB 16 B -0.0%
hits_68.vortex 1.0 vortex-file-compressed 121.95 MB 121.95 MB 16 B -0.0%
hits_18.vortex 1.0 vortex-file-compressed 103.45 MB 103.45 MB 16 B -0.0%
hits_65.vortex 1.0 vortex-file-compressed 184.24 MB 184.24 MB 32 B -0.0%
hits_62.vortex 1.0 vortex-file-compressed 116.56 MB 116.56 MB 32 B -0.0%
hits_81.vortex 1.0 vortex-file-compressed 99.37 MB 99.37 MB 32 B -0.0%
hits_12.vortex 1.0 vortex-file-compressed 99.10 MB 99.10 MB 32 B -0.0%
hits_69.vortex 1.0 vortex-file-compressed 122.65 MB 122.65 MB 48 B -0.0%
hits_76.vortex 1.0 vortex-file-compressed 112.83 MB 112.83 MB 48 B -0.0%
hits_42.vortex 1.0 vortex-file-compressed 223.02 MB 223.02 MB 96 B -0.0%
hits_90.vortex 1.0 vortex-file-compressed 141.48 MB 141.48 MB 64 B -0.0%
hits_86.vortex 1.0 vortex-file-compressed 67.09 MB 67.09 MB 32 B -0.0%
hits_57.vortex 1.0 vortex-file-compressed 127.67 MB 127.67 MB 64 B -0.0%
hits_99.vortex 1.0 vortex-file-compressed 122.33 MB 122.33 MB 64 B -0.0%
hits_48.vortex 1.0 vortex-file-compressed 26.53 MB 26.53 MB 16 B -0.0%
hits_92.vortex 1.0 vortex-file-compressed 146.58 MB 146.58 MB 96 B -0.0%
hits_28.vortex 1.0 vortex-file-compressed 118.35 MB 118.35 MB 80 B -0.0%
hits_93.vortex 1.0 vortex-file-compressed 89.67 MB 89.67 MB 64 B -0.0%
hits_73.vortex 1.0 vortex-file-compressed 108.93 MB 108.93 MB 80 B -0.0%
hits_38.vortex 1.0 vortex-file-compressed 97.80 MB 97.80 MB 96 B -0.0%
hits_35.vortex 1.0 vortex-file-compressed 113.91 MB 113.91 MB 112 B -0.0%
hits_23.vortex 1.0 vortex-file-compressed 75.17 MB 75.17 MB 80 B -0.0%
hits_50.vortex 1.0 vortex-file-compressed 178.87 MB 178.87 MB 208 B -0.0%
hits_46.vortex 1.0 vortex-file-compressed 67.63 MB 67.63 MB 80 B -0.0%
hits_80.vortex 1.0 vortex-file-compressed 104.26 MB 104.26 MB 128 B -0.0%
hits_13.vortex 1.0 vortex-file-compressed 96.97 MB 96.97 MB 128 B -0.0%
hits_51.vortex 1.0 vortex-file-compressed 279.37 MB 279.37 MB 400 B -0.0%
hits_82.vortex 1.0 vortex-file-compressed 97.92 MB 97.92 MB 280 B -0.0%
hits_96.vortex 1.0 vortex-file-compressed 135.00 MB 135.00 MB 392 B -0.0%
hits_84.vortex 1.0 vortex-file-compressed 116.44 MB 116.44 MB 352 B -0.0%
hits_27.vortex 1.0 vortex-file-compressed 121.54 MB 121.54 MB 416 B -0.0%
hits_98.vortex 1.0 vortex-file-compressed 117.63 MB 117.63 MB 456 B -0.0%
hits_16.vortex 1.0 vortex-file-compressed 78.75 MB 78.74 MB 336 B -0.0%
hits_6.vortex 1.0 vortex-file-compressed 91.88 MB 91.88 MB 408 B -0.0%
hits_4.vortex 1.0 vortex-file-compressed 107.60 MB 107.60 MB 576 B -0.0%
hits_63.vortex 1.0 vortex-file-compressed 67.27 MB 67.27 MB 408 B -0.0%
hits_11.vortex 1.0 vortex-file-compressed 78.50 MB 78.50 MB 1.02 KB -0.0%
hits_53.vortex 1.0 vortex-file-compressed 84.98 MB 84.98 MB 1.41 KB -0.0%
hits_3.vortex 1.0 vortex-file-compressed 141.50 MB 141.49 MB 2.70 KB -0.0%
hits_33.vortex 1.0 vortex-file-compressed 55.84 MB 55.84 MB 1.09 KB -0.0%
hits_61.vortex 1.0 vortex-file-compressed 100.54 MB 100.54 MB 2.77 KB -0.0%
hits_22.vortex 1.0 vortex-file-compressed 75.93 MB 75.82 MB 119.05 KB -0.2%

Totals:

  • vortex-compact: 7.11 GB → 7.11 GB (0.0%)
  • vortex-file-compressed: 10.90 GB → 10.90 GB (-0.0%)

@github-actions

github-actions Bot commented Sep 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmarks: Clickbench Sorted on NVME 📖

Commits: PR dd8b5948 vs base 792cef74
Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: -1.0%
Engines: DataFusion No clear signal (-0.8%, environment too noisy confidence) · DuckDB No clear signal (-1.2%, environment too noisy confidence)
Vortex (geomean): hot 0.704x ✅ · cold 0.671x ✅
Parquet (geomean): hot 0.711x ✅ · cold 0.770x ✅
Shifts: Parquet (control) -28.9% · Median polish -25.8%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Table cells: Each cell reads PR / base / %diff. The hot and cold columns mark a change as 🔴 slower or 🟢 faster once it clears this suite's threshold; anything smaller is noise here and is left unmarked, as is hot/cold, because a shift in the warm-up ratio is not a win or a loss by itself.
datafusion / vortex-file-compressed / ns (0.797x ✅, 8↑ 1↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
clickbench-sorted_q23/datafusion:vortex-file-compressed 263598108 / 230266049 / +14.5% 🔴 1103033170 / 461020057 / +139.3% 🔴 0.24 / 0.50 / -52.2%
clickbench-sorted_q24/datafusion:vortex-file-compressed 17565256 / 27911398 / -37.1% 🟢 68630738 / 91005767 / -24.6% 🟢 0.26 / 0.31 / -16.6%
clickbench-sorted_q26/datafusion:vortex-file-compressed 16175391 / 23509204 / -31.2% 🟢 69136027 / 90432141 / -23.5% 🟢 0.23 / 0.26 / -10.0%
clickbench-sorted_q36/datafusion:vortex-file-compressed 36076861 / 50452172 / -28.5% 🟢 111675016 / 126284126 / -11.6% 🟢 0.32 / 0.40 / -19.1%
clickbench-sorted_q37/datafusion:vortex-file-compressed 28933492 / 39954728 / -27.6% 🟢 97542534 / 108879588 / -10.4% 🟢 0.30 / 0.37 / -19.2%
clickbench-sorted_q38/datafusion:vortex-file-compressed 41617234 / 51021210 / -18.4% 🟢 117647883 / 110548979 / +6.4% 0.35 / 0.46 / -23.4%
clickbench-sorted_q39/datafusion:vortex-file-compressed 102768677 / 125701280 / -18.2% 🟢 172100574 / 205738696 / -16.3% 🟢 0.60 / 0.61 / -2.3%
clickbench-sorted_q40/datafusion:vortex-file-compressed 17234952 / 20658600 / -16.6% 🟢 83135047 / 82028409 / +1.3% 0.21 / 0.25 / -17.7%
clickbench-sorted_q41/datafusion:vortex-file-compressed 16241100 / 20534354 / -20.9% 🟢 69979013 / 81407857 / -14.0% 🟢 0.23 / 0.25 / -8.0%
clickbench-sorted_q42/datafusion:vortex-file-compressed 14679378 / 16004904 / -8.3% 54551506 / 69845730 / -21.9% 🟢 0.27 / 0.23 / +17.4%
datafusion / vortex-compact / ns (0.635x ✅, 10↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
clickbench-sorted_q23/datafusion:vortex-compact 268584544 / 486919466 / -44.8% 🟢 929577983 / 831476172 / +11.8% 🔴 0.29 / 0.59 / -50.7%
clickbench-sorted_q24/datafusion:vortex-compact 29804575 / 54603540 / -45.4% 🟢 73462416 / 114895592 / -36.1% 🟢 0.41 / 0.48 / -14.6%
clickbench-sorted_q26/datafusion:vortex-compact 32228808 / 54674673 / -41.1% 🟢 77329525 / 115341762 / -33.0% 🟢 0.42 / 0.47 / -12.1%
clickbench-sorted_q36/datafusion:vortex-compact 34809264 / 55761366 / -37.6% 🟢 116546910 / 131605595 / -11.4% 🟢 0.30 / 0.42 / -29.5%
clickbench-sorted_q37/datafusion:vortex-compact 28303541 / 46644820 / -39.3% 🟢 114173310 / 123835199 / -7.8% 0.25 / 0.38 / -34.2%
clickbench-sorted_q38/datafusion:vortex-compact 27946649 / 44712327 / -37.5% 🟢 111581694 / 117047808 / -4.7% 0.25 / 0.38 / -34.4%
clickbench-sorted_q39/datafusion:vortex-compact 66616533 / 102314592 / -34.9% 🟢 149829208 / 179642161 / -16.6% 🟢 0.44 / 0.57 / -21.9%
clickbench-sorted_q40/datafusion:vortex-compact 18693770 / 27258840 / -31.4% 🟢 66104697 / 81389801 / -18.8% 🟢 0.28 / 0.33 / -15.6%
clickbench-sorted_q41/datafusion:vortex-compact 16483492 / 22644604 / -27.2% 🟢 63500441 / 79362266 / -20.0% 🟢 0.26 / 0.29 / -9.0%
clickbench-sorted_q42/datafusion:vortex-compact 12697036 / 16381051 / -22.5% 🟢 48379188 / 67892353 / -28.7% 🟢 0.26 / 0.24 / +8.8%
datafusion / parquet / ns (0.717x ✅, 10↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
clickbench-sorted_q23/datafusion:parquet 1309433684 / 2508547516 / -47.8% 🟢 1704715550 / 2963520272 / -42.5% 🟢 0.77 / 0.85 / -9.3%
clickbench-sorted_q24/datafusion:parquet 10638340 / 17034666 / -37.5% 🟢 93656659 / 120153456 / -22.1% 🟢 0.11 / 0.14 / -19.9%
clickbench-sorted_q26/datafusion:parquet 11148586 / 16427586 / -32.1% 🟢 90306714 / 115958593 / -22.1% 🟢 0.12 / 0.14 / -12.9%
clickbench-sorted_q36/datafusion:parquet 109896214 / 157039270 / -30.0% 🟢 210604421 / 288565486 / -27.0% 🟢 0.52 / 0.54 / -4.1%
clickbench-sorted_q37/datafusion:parquet 75251786 / 92892207 / -19.0% 🟢 168459488 / 209096835 / -19.4% 🟢 0.45 / 0.44 / +0.6%
clickbench-sorted_q38/datafusion:parquet 102275324 / 141518611 / -27.7% 🟢 202840911 / 247547479 / -18.1% 🟢 0.50 / 0.57 / -11.8%
clickbench-sorted_q39/datafusion:parquet 192075650 / 266882233 / -28.0% 🟢 301450263 / 394197900 / -23.5% 🟢 0.64 / 0.68 / -5.9%
clickbench-sorted_q40/datafusion:parquet 48936390 / 58359787 / -16.1% 🟢 144941011 / 173130588 / -16.3% 🟢 0.34 / 0.34 / +0.2%
clickbench-sorted_q41/datafusion:parquet 47266670 / 58316572 / -18.9% 🟢 140977632 / 170627123 / -17.4% 🟢 0.34 / 0.34 / -1.9%
clickbench-sorted_q42/datafusion:parquet 28594158 / 35504638 / -19.5% 🟢 108387450 / 135214624 / -19.8% 🟢 0.26 / 0.26 / +0.5%
duckdb / vortex-file-compressed / ns (0.716x ✅, 10↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
clickbench-sorted_q23/duckdb:vortex-file-compressed 57938013 / 115802332 / -50.0% 🟢 111717690 / 200107936 / -44.2% 🟢 0.52 / 0.58 / -10.4%
clickbench-sorted_q24/duckdb:vortex-file-compressed 20011356 / 30174905 / -33.7% 🟢 36490678 / 77821228 / -53.1% 🟢 0.55 / 0.39 / +41.4%
clickbench-sorted_q26/duckdb:vortex-file-compressed 16031190 / 24864487 / -35.5% 🟢 30532686 / 72539562 / -57.9% 🟢 0.53 / 0.34 / +53.2%
clickbench-sorted_q36/duckdb:vortex-file-compressed 54229422 / 70666896 / -23.3% 🟢 118896907 / 188202509 / -36.8% 🟢 0.46 / 0.38 / +21.5%
clickbench-sorted_q37/duckdb:vortex-file-compressed 47977593 / 62554466 / -23.3% 🟢 91624296 / 173725299 / -47.3% 🟢 0.52 / 0.36 / +45.4%
clickbench-sorted_q38/duckdb:vortex-file-compressed 58639110 / 73483255 / -20.2% 🟢 111251712 / 184361770 / -39.7% 🟢 0.53 / 0.40 / +32.2%
clickbench-sorted_q39/duckdb:vortex-file-compressed 145811460 / 192459156 / -24.2% 🟢 217052324 / 294192835 / -26.2% 🟢 0.67 / 0.65 / +2.7%
clickbench-sorted_q40/duckdb:vortex-file-compressed 24932751 / 32142184 / -22.4% 🟢 60870968 / 120284377 / -49.4% 🟢 0.41 / 0.27 / +53.3%
clickbench-sorted_q41/duckdb:vortex-file-compressed 25426550 / 32315960 / -21.3% 🟢 60356232 / 121494475 / -50.3% 🟢 0.42 / 0.27 / +58.4%
clickbench-sorted_q42/duckdb:vortex-file-compressed 21896971 / 28825318 / -24.0% 🟢 41014883 / 104691430 / -60.8% 🟢 0.53 / 0.28 / +93.9%
duckdb / vortex-compact / ns (0.677x ✅, 10↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
clickbench-sorted_q23/duckdb:vortex-compact 58871661 / 110740048 / -46.8% 🟢 99916178 / 193671621 / -48.4% 🟢 0.59 / 0.57 / +3.0%
clickbench-sorted_q24/duckdb:vortex-compact 22413513 / 32733526 / -31.5% 🟢 33696103 / 85544947 / -60.6% 🟢 0.67 / 0.38 / +73.8%
clickbench-sorted_q26/duckdb:vortex-compact 17299662 / 30264396 / -42.8% 🟢 30098493 / 88621818 / -66.0% 🟢 0.57 / 0.34 / +68.3%
clickbench-sorted_q36/duckdb:vortex-compact 48907236 / 72717776 / -32.7% 🟢 113530564 / 194967444 / -41.8% 🟢 0.43 / 0.37 / +15.5%
clickbench-sorted_q37/duckdb:vortex-compact 41917326 / 59278560 / -29.3% 🟢 110222742 / 169709302 / -35.1% 🟢 0.38 / 0.35 / +8.9%
clickbench-sorted_q38/duckdb:vortex-compact 43191239 / 63017914 / -31.5% 🟢 107133011 / 188302421 / -43.1% 🟢 0.40 / 0.33 / +20.5%
clickbench-sorted_q39/duckdb:vortex-compact 103672126 / 124986676 / -17.1% 🟢 182347698 / 256092731 / -28.8% 🟢 0.57 / 0.49 / +16.5%
clickbench-sorted_q40/duckdb:vortex-compact 23633252 / 33186037 / -28.8% 🟢 52794134 / 127433509 / -58.6% 🟢 0.45 / 0.26 / +71.9%
clickbench-sorted_q41/duckdb:vortex-compact 24659578 / 33638280 / -26.7% 🟢 54586940 / 126586425 / -56.9% 🟢 0.45 / 0.27 / +70.0%
clickbench-sorted_q42/duckdb:vortex-compact 18957256 / 27582510 / -31.3% 🟢 39228059 / 99778500 / -60.7% 🟢 0.48 / 0.28 / +74.8%
duckdb / parquet / ns (0.705x ✅, 10↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
clickbench-sorted_q23/duckdb:parquet 144545840 / 226608774 / -36.2% 🟢 173173856 / 253394212 / -31.7% 🟢 0.83 / 0.89 / -6.7%
clickbench-sorted_q24/duckdb:parquet 19279350 / 30326583 / -36.4% 🟢 41375343 / 54727313 / -24.4% 🟢 0.47 / 0.55 / -15.9%
clickbench-sorted_q26/duckdb:parquet 14451060 / 27649460 / -47.7% 🟢 27734885 / 44321046 / -37.4% 🟢 0.52 / 0.62 / -16.5%
clickbench-sorted_q36/duckdb:parquet 54155526 / 72655658 / -25.5% 🟢 87865655 / 99559062 / -11.7% 🟢 0.62 / 0.73 / -15.5%
clickbench-sorted_q37/duckdb:parquet 40888558 / 61362932 / -33.4% 🟢 65983087 / 92172378 / -28.4% 🟢 0.62 / 0.67 / -6.9%
clickbench-sorted_q38/duckdb:parquet 48401544 / 61961824 / -21.9% 🟢 83688828 / 94464780 / -11.4% 🟢 0.58 / 0.66 / -11.8%
clickbench-sorted_q39/duckdb:parquet 85657194 / 112055923 / -23.6% 🟢 135403932 / 164200107 / -17.5% 🟢 0.63 / 0.68 / -7.3%
clickbench-sorted_q40/duckdb:parquet 21676736 / 31234602 / -30.6% 🟢 56834730 / 73834150 / -23.0% 🟢 0.38 / 0.42 / -9.8%
clickbench-sorted_q41/duckdb:parquet 25163733 / 29645792 / -15.1% 🟢 59617442 / 66440846 / -10.3% 🟢 0.42 / 0.45 / -5.4%
clickbench-sorted_q42/duckdb:parquet 16828763 / 20640986 / -18.5% 🟢 32401864 / 43943230 / -26.3% 🟢 0.52 / 0.47 / +10.6%
File Size Changes (200 files changed, -0.0% overall, 100↑ 100↓)
File Scale Format Base HEAD Change %
hits_061.vortex 1.0 vortex-compact 123.47 MB 125.24 MB +1.78 MB +1.4%
hits_000.vortex 1.0 vortex-compact 98.20 MB 99.56 MB +1.36 MB +1.4%
hits_065.vortex 1.0 vortex-compact 124.17 MB 125.85 MB +1.69 MB +1.4%
hits_089.vortex 1.0 vortex-compact 98.77 MB 100.10 MB +1.33 MB +1.4%
hits_093.vortex 1.0 vortex-compact 97.63 MB 98.85 MB +1.22 MB +1.3%
hits_002.vortex 1.0 vortex-compact 124.15 MB 125.69 MB +1.54 MB +1.2%
hits_076.vortex 1.0 vortex-file-compressed 158.47 MB 160.02 MB +1.55 MB +1.0%
hits_023.vortex 1.0 vortex-file-compressed 194.38 MB 196.00 MB +1.62 MB +0.8%
hits_030.vortex 1.0 vortex-compact 98.03 MB 98.79 MB +781.21 KB +0.8%
hits_086.vortex 1.0 vortex-compact 139.63 MB 140.69 MB +1.06 MB +0.8%
hits_041.vortex 1.0 vortex-compact 97.10 MB 97.83 MB +742.53 KB +0.7%
hits_018.vortex 1.0 vortex-file-compressed 197.59 MB 199.05 MB +1.46 MB +0.7%
hits_074.vortex 1.0 vortex-file-compressed 197.26 MB 198.41 MB +1.15 MB +0.6%
hits_098.vortex 1.0 vortex-compact 103.76 MB 104.25 MB +500.20 KB +0.5%
hits_004.vortex 1.0 vortex-file-compressed 130.47 MB 131.07 MB +616.06 KB +0.5%
hits_039.vortex 1.0 vortex-compact 124.33 MB 124.88 MB +556.68 KB +0.4%
hits_082.vortex 1.0 vortex-compact 105.79 MB 106.23 MB +449.09 KB +0.4%
hits_019.vortex 1.0 vortex-compact 106.18 MB 106.61 MB +441.82 KB +0.4%
hits_077.vortex 1.0 vortex-file-compressed 171.58 MB 172.21 MB +647.23 KB +0.4%
hits_064.vortex 1.0 vortex-compact 140.14 MB 140.62 MB +497.03 KB +0.3%
hits_085.vortex 1.0 vortex-file-compressed 198.93 MB 199.58 MB +664.43 KB +0.3%
hits_050.vortex 1.0 vortex-file-compressed 160.01 MB 160.51 MB +520.04 KB +0.3%
hits_013.vortex 1.0 vortex-compact 125.31 MB 125.69 MB +394.41 KB +0.3%
hits_010.vortex 1.0 vortex-compact 123.31 MB 123.67 MB +370.77 KB +0.3%
hits_095.vortex 1.0 vortex-file-compressed 153.38 MB 153.80 MB +431.92 KB +0.3%
hits_062.vortex 1.0 vortex-file-compressed 169.98 MB 170.44 MB +466.75 KB +0.3%
hits_021.vortex 1.0 vortex-file-compressed 153.46 MB 153.84 MB +387.72 KB +0.2%
hits_034.vortex 1.0 vortex-file-compressed 180.94 MB 181.31 MB +380.29 KB +0.2%
hits_071.vortex 1.0 vortex-compact 105.55 MB 105.76 MB +219.99 KB +0.2%
hits_043.vortex 1.0 vortex-compact 94.26 MB 94.44 MB +186.85 KB +0.2%
hits_075.vortex 1.0 vortex-file-compressed 188.28 MB 188.57 MB +295.09 KB +0.2%
hits_001.vortex 1.0 vortex-file-compressed 188.08 MB 188.36 MB +291.34 KB +0.2%
hits_015.vortex 1.0 vortex-compact 98.24 MB 98.38 MB +149.87 KB +0.1%
hits_073.vortex 1.0 vortex-file-compressed 172.09 MB 172.33 MB +252.84 KB +0.1%
hits_094.vortex 1.0 vortex-file-compressed 157.81 MB 158.03 MB +223.77 KB +0.1%
hits_074.vortex 1.0 vortex-compact 152.92 MB 153.12 MB +207.12 KB +0.1%
hits_031.vortex 1.0 vortex-compact 111.55 MB 111.70 MB +150.88 KB +0.1%
hits_065.vortex 1.0 vortex-file-compressed 160.10 MB 160.31 MB +216.48 KB +0.1%
hits_016.vortex 1.0 vortex-file-compressed 179.60 MB 179.82 MB +217.42 KB +0.1%
hits_009.vortex 1.0 vortex-file-compressed 100.73 MB 100.84 MB +113.80 KB +0.1%
hits_079.vortex 1.0 vortex-file-compressed 180.43 MB 180.63 MB +200.09 KB +0.1%
hits_016.vortex 1.0 vortex-compact 127.39 MB 127.53 MB +138.63 KB +0.1%
hits_070.vortex 1.0 vortex-compact 153.62 MB 153.77 MB +162.77 KB +0.1%
hits_025.vortex 1.0 vortex-file-compressed 171.16 MB 171.33 MB +179.00 KB +0.1%
hits_029.vortex 1.0 vortex-compact 153.48 MB 153.63 MB +157.57 KB +0.1%
hits_044.vortex 1.0 vortex-file-compressed 199.40 MB 199.58 MB +187.19 KB +0.1%
hits_052.vortex 1.0 vortex-file-compressed 129.89 MB 130.00 MB +119.48 KB +0.1%
hits_081.vortex 1.0 vortex-compact 153.58 MB 153.71 MB +133.59 KB +0.1%
hits_095.vortex 1.0 vortex-compact 110.53 MB 110.62 MB +95.11 KB +0.1%
hits_014.vortex 1.0 vortex-compact 133.74 MB 133.86 MB +114.01 KB +0.1%
hits_099.vortex 1.0 vortex-file-compressed 170.55 MB 170.69 MB +141.82 KB +0.1%
hits_096.vortex 1.0 vortex-compact 153.11 MB 153.23 MB +124.60 KB +0.1%
hits_042.vortex 1.0 vortex-compact 136.26 MB 136.37 MB +110.71 KB +0.1%
hits_028.vortex 1.0 vortex-compact 115.68 MB 115.77 MB +89.33 KB +0.1%
hits_061.vortex 1.0 vortex-file-compressed 159.79 MB 159.90 MB +113.04 KB +0.1%
hits_085.vortex 1.0 vortex-compact 153.33 MB 153.43 MB +105.05 KB +0.1%
hits_057.vortex 1.0 vortex-compact 111.73 MB 111.81 MB +75.59 KB +0.1%
hits_002.vortex 1.0 vortex-file-compressed 159.94 MB 160.04 MB +105.97 KB +0.1%
hits_024.vortex 1.0 vortex-file-compressed 159.46 MB 159.56 MB +103.50 KB +0.1%
hits_011.vortex 1.0 vortex-compact 152.78 MB 152.87 MB +90.98 KB +0.1%
hits_099.vortex 1.0 vortex-compact 133.06 MB 133.13 MB +73.76 KB +0.1%
hits_030.vortex 1.0 vortex-file-compressed 130.89 MB 130.96 MB +71.88 KB +0.1%
hits_091.vortex 1.0 vortex-file-compressed 146.02 MB 146.10 MB +76.70 KB +0.1%
hits_093.vortex 1.0 vortex-file-compressed 130.80 MB 130.87 MB +65.34 KB +0.0%
hits_073.vortex 1.0 vortex-compact 134.09 MB 134.15 MB +62.16 KB +0.0%
hits_079.vortex 1.0 vortex-compact 128.04 MB 128.09 MB +57.79 KB +0.0%
hits_008.vortex 1.0 vortex-compact 105.70 MB 105.74 MB +46.08 KB +0.0%
hits_046.vortex 1.0 vortex-compact 74.97 MB 75.00 MB +31.64 KB +0.0%
hits_072.vortex 1.0 vortex-compact 75.55 MB 75.57 MB +26.48 KB +0.0%
hits_035.vortex 1.0 vortex-compact 75.91 MB 75.94 MB +26.59 KB +0.0%
hits_049.vortex 1.0 vortex-compact 140.49 MB 140.53 MB +45.17 KB +0.0%
hits_090.vortex 1.0 vortex-compact 141.21 MB 141.26 MB +44.86 KB +0.0%
hits_066.vortex 1.0 vortex-file-compressed 162.38 MB 162.43 MB +50.38 KB +0.0%
hits_010.vortex 1.0 vortex-file-compressed 168.04 MB 168.08 MB +46.16 KB +0.0%
hits_078.vortex 1.0 vortex-compact 97.77 MB 97.79 MB +22.02 KB +0.0%
hits_007.vortex 1.0 vortex-file-compressed 199.47 MB 199.51 MB +43.44 KB +0.0%
hits_044.vortex 1.0 vortex-compact 153.60 MB 153.63 MB +33.34 KB +0.0%
hits_038.vortex 1.0 vortex-compact 139.54 MB 139.57 MB +29.82 KB +0.0%
hits_077.vortex 1.0 vortex-compact 134.18 MB 134.20 MB +27.45 KB +0.0%
hits_097.vortex 1.0 vortex-file-compressed 191.86 MB 191.90 MB +39.05 KB +0.0%
hits_040.vortex 1.0 vortex-compact 109.95 MB 109.97 MB +20.67 KB +0.0%
hits_082.vortex 1.0 vortex-file-compressed 138.57 MB 138.59 MB +25.56 KB +0.0%
hits_035.vortex 1.0 vortex-file-compressed 102.36 MB 102.38 MB +17.67 KB +0.0%
hits_003.vortex 1.0 vortex-compact 104.57 MB 104.58 MB +18.01 KB +0.0%
hits_012.vortex 1.0 vortex-file-compressed 190.00 MB 190.03 MB +27.31 KB +0.0%
hits_026.vortex 1.0 vortex-file-compressed 130.34 MB 130.36 MB +17.97 KB +0.0%
hits_084.vortex 1.0 vortex-file-compressed 153.70 MB 153.72 MB +19.93 KB +0.0%
hits_072.vortex 1.0 vortex-file-compressed 101.64 MB 101.65 MB +12.84 KB +0.0%
hits_053.vortex 1.0 vortex-compact 139.01 MB 139.03 MB +15.02 KB +0.0%
hits_084.vortex 1.0 vortex-compact 110.64 MB 110.65 MB +11.53 KB +0.0%
hits_096.vortex 1.0 vortex-file-compressed 198.85 MB 198.87 MB +18.30 KB +0.0%
hits_033.vortex 1.0 vortex-compact 153.50 MB 153.51 MB +14.03 KB +0.0%
hits_068.vortex 1.0 vortex-compact 112.49 MB 112.50 MB +9.53 KB +0.0%
hits_021.vortex 1.0 vortex-compact 110.62 MB 110.62 MB +6.37 KB +0.0%
hits_011.vortex 1.0 vortex-file-compressed 198.52 MB 198.52 MB +8.05 KB +0.0%
hits_008.vortex 1.0 vortex-file-compressed 138.79 MB 138.79 MB +5.19 KB +0.0%
hits_062.vortex 1.0 vortex-compact 132.48 MB 132.49 MB +4.51 KB +0.0%
hits_083.vortex 1.0 vortex-file-compressed 156.20 MB 156.20 MB +5.02 KB +0.0%
hits_017.vortex 1.0 vortex-file-compressed 145.77 MB 145.78 MB +1.53 KB +0.0%
hits_032.vortex 1.0 vortex-compact 110.61 MB 110.61 MB +920 B +0.0%
hits_078.vortex 1.0 vortex-file-compressed 130.10 MB 130.10 MB 2.46 KB -0.0%
hits_046.vortex 1.0 vortex-file-compressed 100.51 MB 100.51 MB 2.20 KB -0.0%
hits_009.vortex 1.0 vortex-compact 75.21 MB 75.21 MB 2.09 KB -0.0%
hits_050.vortex 1.0 vortex-compact 125.37 MB 125.36 MB 6.72 KB -0.0%
hits_006.vortex 1.0 vortex-file-compressed 125.64 MB 125.63 MB 9.30 KB -0.0%
hits_036.vortex 1.0 vortex-compact 133.35 MB 133.34 MB 10.71 KB -0.0%
hits_080.vortex 1.0 vortex-file-compressed 126.19 MB 126.18 MB 10.66 KB -0.0%
hits_088.vortex 1.0 vortex-file-compressed 171.36 MB 171.34 MB 14.81 KB -0.0%
hits_042.vortex 1.0 vortex-file-compressed 200.61 MB 200.59 MB 19.60 KB -0.0%
hits_047.vortex 1.0 vortex-compact 110.48 MB 110.47 MB 15.26 KB -0.0%
hits_034.vortex 1.0 vortex-compact 133.37 MB 133.35 MB 20.79 KB -0.0%
hits_019.vortex 1.0 vortex-file-compressed 139.69 MB 139.67 MB 24.39 KB -0.0%
hits_020.vortex 1.0 vortex-compact 111.66 MB 111.64 MB 19.53 KB -0.0%
hits_088.vortex 1.0 vortex-compact 133.96 MB 133.94 MB 24.12 KB -0.0%
hits_094.vortex 1.0 vortex-compact 110.70 MB 110.67 MB 22.52 KB -0.0%
hits_063.vortex 1.0 vortex-file-compressed 130.77 MB 130.74 MB 27.29 KB -0.0%
hits_060.vortex 1.0 vortex-compact 141.18 MB 141.15 MB 32.82 KB -0.0%
hits_067.vortex 1.0 vortex-compact 98.45 MB 98.42 MB 24.91 KB -0.0%
hits_055.vortex 1.0 vortex-compact 153.43 MB 153.39 MB 39.12 KB -0.0%
hits_001.vortex 1.0 vortex-compact 139.00 MB 138.96 MB 36.08 KB -0.0%
hits_036.vortex 1.0 vortex-file-compressed 170.71 MB 170.67 MB 46.09 KB -0.0%
hits_058.vortex 1.0 vortex-file-compressed 154.16 MB 154.12 MB 43.30 KB -0.0%
hits_005.vortex 1.0 vortex-file-compressed 166.47 MB 166.42 MB 48.89 KB -0.0%
hits_025.vortex 1.0 vortex-compact 133.47 MB 133.43 MB 41.42 KB -0.0%
hits_087.vortex 1.0 vortex-file-compressed 159.93 MB 159.87 MB 54.51 KB -0.0%
hits_038.vortex 1.0 vortex-file-compressed 189.15 MB 189.08 MB 69.02 KB -0.0%
hits_069.vortex 1.0 vortex-file-compressed 141.66 MB 141.60 MB 58.72 KB -0.0%
hits_083.vortex 1.0 vortex-compact 110.08 MB 110.03 MB 45.95 KB -0.0%
hits_045.vortex 1.0 vortex-file-compressed 138.78 MB 138.72 MB 58.01 KB -0.0%
hits_041.vortex 1.0 vortex-file-compressed 130.25 MB 130.19 MB 61.96 KB -0.0%
hits_047.vortex 1.0 vortex-file-compressed 153.61 MB 153.53 MB 76.41 KB -0.0%
hits_091.vortex 1.0 vortex-compact 112.26 MB 112.20 MB 58.50 KB -0.1%
hits_092.vortex 1.0 vortex-compact 153.50 MB 153.42 MB 81.44 KB -0.1%
hits_090.vortex 1.0 vortex-file-compressed 191.56 MB 191.46 MB 102.05 KB -0.1%
hits_023.vortex 1.0 vortex-compact 142.68 MB 142.60 MB 80.76 KB -0.1%
hits_022.vortex 1.0 vortex-compact 153.11 MB 153.03 MB 87.08 KB -0.1%
hits_031.vortex 1.0 vortex-file-compressed 158.94 MB 158.83 MB 110.48 KB -0.1%
hits_032.vortex 1.0 vortex-file-compressed 153.82 MB 153.71 MB 116.18 KB -0.1%
hits_037.vortex 1.0 vortex-file-compressed 176.57 MB 176.44 MB 138.68 KB -0.1%
hits_045.vortex 1.0 vortex-compact 106.14 MB 106.06 MB 86.09 KB -0.1%
hits_060.vortex 1.0 vortex-file-compressed 192.20 MB 192.05 MB 156.31 KB -0.1%
hits_048.vortex 1.0 vortex-file-compressed 198.89 MB 198.72 MB 175.59 KB -0.1%
hits_007.vortex 1.0 vortex-compact 153.94 MB 153.81 MB 135.95 KB -0.1%
hits_018.vortex 1.0 vortex-compact 152.88 MB 152.74 MB 135.92 KB -0.1%
hits_054.vortex 1.0 vortex-compact 112.30 MB 112.20 MB 99.88 KB -0.1%
hits_059.vortex 1.0 vortex-compact 153.09 MB 152.95 MB 137.39 KB -0.1%
hits_020.vortex 1.0 vortex-file-compressed 159.36 MB 159.19 MB 166.36 KB -0.1%
hits_003.vortex 1.0 vortex-file-compressed 135.78 MB 135.64 MB 143.18 KB -0.1%
hits_055.vortex 1.0 vortex-file-compressed 199.18 MB 198.95 MB 238.73 KB -0.1%
hits_051.vortex 1.0 vortex-file-compressed 172.43 MB 172.22 MB 215.98 KB -0.1%
hits_049.vortex 1.0 vortex-file-compressed 190.52 MB 190.26 MB 264.52 KB -0.1%
hits_026.vortex 1.0 vortex-compact 99.01 MB 98.87 MB 144.52 KB -0.1%
hits_057.vortex 1.0 vortex-file-compressed 159.21 MB 158.98 MB 242.30 KB -0.1%
hits_048.vortex 1.0 vortex-compact 152.78 MB 152.54 MB 245.22 KB -0.2%
hits_043.vortex 1.0 vortex-file-compressed 125.93 MB 125.73 MB 208.55 KB -0.2%
hits_012.vortex 1.0 vortex-compact 140.02 MB 139.77 MB 255.78 KB -0.2%
hits_052.vortex 1.0 vortex-compact 98.46 MB 98.28 MB 187.21 KB -0.2%
hits_089.vortex 1.0 vortex-file-compressed 130.95 MB 130.69 MB 267.95 KB -0.2%
hits_015.vortex 1.0 vortex-file-compressed 130.46 MB 130.20 MB 269.40 KB -0.2%
hits_033.vortex 1.0 vortex-file-compressed 199.66 MB 199.25 MB 420.56 KB -0.2%
hits_069.vortex 1.0 vortex-compact 99.70 MB 99.48 MB 226.94 KB -0.2%
hits_029.vortex 1.0 vortex-file-compressed 199.65 MB 199.19 MB 474.93 KB -0.2%
hits_059.vortex 1.0 vortex-file-compressed 198.84 MB 198.37 MB 482.82 KB -0.2%
hits_070.vortex 1.0 vortex-file-compressed 199.66 MB 199.18 MB 491.43 KB -0.2%
hits_067.vortex 1.0 vortex-file-compressed 131.20 MB 130.86 MB 350.34 KB -0.3%
hits_080.vortex 1.0 vortex-compact 96.61 MB 96.34 MB 272.08 KB -0.3%
hits_000.vortex 1.0 vortex-file-compressed 131.10 MB 130.73 MB 372.58 KB -0.3%
hits_051.vortex 1.0 vortex-compact 134.22 MB 133.83 MB 400.98 KB -0.3%
hits_017.vortex 1.0 vortex-compact 112.48 MB 112.13 MB 362.89 KB -0.3%
hits_098.vortex 1.0 vortex-file-compressed 137.39 MB 136.94 MB 464.36 KB -0.3%
hits_068.vortex 1.0 vortex-file-compressed 159.88 MB 159.28 MB 610.62 KB -0.4%
hits_022.vortex 1.0 vortex-file-compressed 198.42 MB 197.68 MB 758.77 KB -0.4%
hits_005.vortex 1.0 vortex-compact 116.33 MB 115.88 MB 455.49 KB -0.4%
hits_014.vortex 1.0 vortex-file-compressed 172.02 MB 171.30 MB 737.37 KB -0.4%
hits_054.vortex 1.0 vortex-file-compressed 146.77 MB 146.15 MB 636.22 KB -0.4%
hits_028.vortex 1.0 vortex-file-compressed 151.58 MB 150.93 MB 660.89 KB -0.4%
hits_006.vortex 1.0 vortex-compact 96.27 MB 95.86 MB 425.46 KB -0.4%
hits_056.vortex 1.0 vortex-compact 101.29 MB 100.85 MB 448.20 KB -0.4%
hits_066.vortex 1.0 vortex-compact 118.12 MB 117.53 MB 603.89 KB -0.5%
hits_040.vortex 1.0 vortex-file-compressed 142.73 MB 142.00 MB 745.93 KB -0.5%
hits_092.vortex 1.0 vortex-file-compressed 199.15 MB 198.09 MB 1.06 MB -0.5%
hits_071.vortex 1.0 vortex-file-compressed 139.42 MB 138.68 MB 760.79 KB -0.5%
hits_037.vortex 1.0 vortex-compact 132.88 MB 132.15 MB 738.23 KB -0.5%
hits_056.vortex 1.0 vortex-file-compressed 135.23 MB 134.49 MB 757.90 KB -0.5%
hits_087.vortex 1.0 vortex-compact 125.77 MB 125.07 MB 708.76 KB -0.6%
hits_053.vortex 1.0 vortex-file-compressed 190.02 MB 188.97 MB 1.05 MB -0.6%
hits_081.vortex 1.0 vortex-file-compressed 199.29 MB 198.18 MB 1.10 MB -0.6%
hits_058.vortex 1.0 vortex-compact 111.75 MB 111.05 MB 714.52 KB -0.6%
hits_076.vortex 1.0 vortex-compact 125.51 MB 124.55 MB 984.37 KB -0.8%
hits_024.vortex 1.0 vortex-compact 124.97 MB 124.00 MB 1001.42 KB -0.8%
hits_075.vortex 1.0 vortex-compact 138.98 MB 137.83 MB 1.14 MB -0.8%
hits_027.vortex 1.0 vortex-compact 139.29 MB 138.05 MB 1.24 MB -0.9%
hits_039.vortex 1.0 vortex-file-compressed 160.68 MB 159.15 MB 1.53 MB -0.9%
hits_013.vortex 1.0 vortex-file-compressed 160.32 MB 158.77 MB 1.55 MB -1.0%
hits_086.vortex 1.0 vortex-file-compressed 193.18 MB 191.18 MB 2.00 MB -1.0%
hits_063.vortex 1.0 vortex-compact 98.94 MB 97.83 MB 1.11 MB -1.1%
hits_097.vortex 1.0 vortex-compact 141.89 MB 140.24 MB 1.65 MB -1.2%
hits_027.vortex 1.0 vortex-file-compressed 191.13 MB 188.81 MB 2.32 MB -1.2%
hits_004.vortex 1.0 vortex-compact 98.30 MB 97.04 MB 1.26 MB -1.3%
hits_064.vortex 1.0 vortex-file-compressed 193.54 MB 190.46 MB 3.08 MB -1.6%

Totals:

  • vortex-compact: 11.93 GB → 11.93 GB (+0.0%)
  • vortex-file-compressed: 15.88 GB → 15.87 GB (-0.1%)

@github-actions

github-actions Bot commented Sep 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-H SF=10 on NVME 📖

Commits: PR dd8b5948 vs base 792cef74
Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: -13.0%
Engines: DataFusion No clear signal (-2.9%, environment too noisy confidence) · DuckDB No clear signal (-22.0%, environment too noisy confidence)
Vortex (geomean): hot 0.660x ✅ · cold 0.646x ✅
Parquet (geomean): hot 0.758x ✅ · cold 0.842x ✅
Shifts: Parquet (control) -24.2% · Median polish -33.6%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Table cells: Each cell reads PR / base / %diff. The hot and cold columns mark a change as 🔴 slower or 🟢 faster once it clears this suite's threshold; anything smaller is noise here and is left unmarked, as is hot/cold, because a shift in the warm-up ratio is not a win or a loss by itself.
datafusion / vortex-file-compressed / ns (0.662x ✅, 22↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/datafusion:vortex-file-compressed 157211562 / 322658511 / -51.3% 🟢 292027243 / 471733355 / -38.1% 🟢 0.54 / 0.68 / -21.3%
tpch_q02/datafusion:vortex-file-compressed 65282858 / 96906925 / -32.6% 🟢 101381884 / 175392103 / -42.2% 🟢 0.64 / 0.55 / +16.5%
tpch_q03/datafusion:vortex-file-compressed 86066316 / 137749688 / -37.5% 🟢 187797307 / 275420380 / -31.8% 🟢 0.46 / 0.50 / -8.4%
tpch_q04/datafusion:vortex-file-compressed 41320568 / 58772273 / -29.7% 🟢 105213897 / 129475252 / -18.7% 🟢 0.39 / 0.45 / -13.5%
tpch_q05/datafusion:vortex-file-compressed 164434166 / 264415682 / -37.8% 🟢 265568490 / 457894247 / -42.0% 🟢 0.62 / 0.58 / +7.2%
tpch_q06/datafusion:vortex-file-compressed 19653494 / 22974734 / -14.5% 🟢 128712211 / 130433788 / -1.3% 0.15 / 0.18 / -13.3%
tpch_q07/datafusion:vortex-file-compressed 201660964 / 344429826 / -41.5% 🟢 310072979 / 505949028 / -38.7% 🟢 0.65 / 0.68 / -4.5%
tpch_q08/datafusion:vortex-file-compressed 171136799 / 265041065 / -35.4% 🟢 301392604 / 442736622 / -31.9% 🟢 0.57 / 0.60 / -5.1%
tpch_q09/datafusion:vortex-file-compressed 272710817 / 446768536 / -39.0% 🟢 451646979 / 628641102 / -28.2% 🟢 0.60 / 0.71 / -15.0%
tpch_q10/datafusion:vortex-file-compressed 103333495 / 161575605 / -36.0% 🟢 197712651 / 303682044 / -34.9% 🟢 0.52 / 0.53 / -1.8%
tpch_q11/datafusion:vortex-file-compressed 44990138 / 68650255 / -34.5% 🟢 79958448 / 135921470 / -41.2% 🟢 0.56 / 0.51 / +11.4%
tpch_q12/datafusion:vortex-file-compressed 55060031 / 77268575 / -28.7% 🟢 130358579 / 182393249 / -28.5% 🟢 0.42 / 0.42 / -0.3%
tpch_q13/datafusion:vortex-file-compressed 78088246 / 115702043 / -32.5% 🟢 131983580 / 193668715 / -31.9% 🟢 0.59 / 0.60 / -1.0%
tpch_q14/datafusion:vortex-file-compressed 26659201 / 37835665 / -29.5% 🟢 154826744 / 172798186 / -10.4% 🟢 0.17 / 0.22 / -21.4%
tpch_q15/datafusion:vortex-file-compressed 46338722 / 68010467 / -31.9% 🟢 162786141 / 195734926 / -16.8% 🟢 0.28 / 0.35 / -18.1%
tpch_q16/datafusion:vortex-file-compressed 40554415 / 60298487 / -32.7% 🟢 78997025 / 138449486 / -42.9% 🟢 0.51 / 0.44 / +17.9%
tpch_q17/datafusion:vortex-file-compressed 272466075 / 410486659 / -33.6% 🟢 346325714 / 565669197 / -38.8% 🟢 0.79 / 0.73 / +8.4%
tpch_q18/datafusion:vortex-file-compressed 350507667 / 603890184 / -42.0% 🟢 416731431 / 775046249 / -46.2% 🟢 0.84 / 0.78 / +7.9%
tpch_q19/datafusion:vortex-file-compressed 46469530 / 54613277 / -14.9% 🟢 165236801 / 172569611 / -4.2% 0.28 / 0.32 / -11.1%
tpch_q20/datafusion:vortex-file-compressed 81136522 / 123086233 / -34.1% 🟢 189045201 / 262770173 / -28.1% 🟢 0.43 / 0.47 / -8.4%
tpch_q21/datafusion:vortex-file-compressed 268591026 / 420353652 / -36.1% 🟢 341309094 / 542508781 / -37.1% 🟢 0.79 / 0.77 / +1.6%
tpch_q22/datafusion:vortex-file-compressed 26660422 / 37057673 / -28.1% 🟢 65348669 / 101879240 / -35.9% 🟢 0.41 / 0.36 / +12.2%
datafusion / vortex-compact / ns (0.617x ✅, 22↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/datafusion:vortex-compact 164778075 / 349705695 / -52.9% 🟢 278881521 / 467358811 / -40.3% 🟢 0.59 / 0.75 / -21.0%
tpch_q02/datafusion:vortex-compact 65839618 / 98827463 / -33.4% 🟢 101675565 / 174632226 / -41.8% 🟢 0.65 / 0.57 / +14.4%
tpch_q03/datafusion:vortex-compact 87944657 / 144574005 / -39.2% 🟢 169719015 / 278568959 / -39.1% 🟢 0.52 / 0.52 / -0.2%
tpch_q04/datafusion:vortex-compact 52991009 / 80757259 / -34.4% 🟢 101723769 / 151116579 / -32.7% 🟢 0.52 / 0.53 / -2.5%
tpch_q05/datafusion:vortex-compact 158149931 / 268236682 / -41.0% 🟢 261761918 / 441889491 / -40.8% 🟢 0.60 / 0.61 / -0.5%
tpch_q06/datafusion:vortex-compact 27771418 / 40461332 / -31.4% 🟢 109303496 / 122857724 / -11.0% 🟢 0.25 / 0.33 / -22.9%
tpch_q07/datafusion:vortex-compact 211218189 / 366089798 / -42.3% 🟢 293099626 / 480205544 / -39.0% 🟢 0.72 / 0.76 / -5.5%
tpch_q08/datafusion:vortex-compact 170982668 / 282775446 / -39.5% 🟢 274132872 / 451803613 / -39.3% 🟢 0.62 / 0.63 / -0.3%
tpch_q09/datafusion:vortex-compact 266098489 / 474619041 / -43.9% 🟢 383867699 / 633700336 / -39.4% 🟢 0.69 / 0.75 / -7.4%
tpch_q10/datafusion:vortex-compact 110902338 / 177356020 / -37.5% 🟢 186524817 / 298188828 / -37.4% 🟢 0.59 / 0.59 / -0.0%
tpch_q11/datafusion:vortex-compact 45719825 / 70420326 / -35.1% 🟢 78550741 / 138711759 / -43.4% 🟢 0.58 / 0.51 / +14.6%
tpch_q12/datafusion:vortex-compact 74731484 / 121927737 / -38.7% 🟢 129323097 / 201757915 / -35.9% 🟢 0.58 / 0.60 / -4.4%
tpch_q13/datafusion:vortex-compact 87600443 / 140273968 / -37.6% 🟢 136259906 / 210130779 / -35.2% 🟢 0.64 / 0.67 / -3.7%
tpch_q14/datafusion:vortex-compact 34227153 / 52093286 / -34.3% 🟢 137718731 / 157050479 / -12.3% 🟢 0.25 / 0.33 / -25.1%
tpch_q15/datafusion:vortex-compact 67166282 / 106992330 / -37.2% 🟢 162002711 / 208983299 / -22.5% 🟢 0.41 / 0.51 / -19.0%
tpch_q16/datafusion:vortex-compact 44060859 / 65742098 / -33.0% 🟢 77967820 / 141472705 / -44.9% 🟢 0.57 / 0.46 / +21.6%
tpch_q17/datafusion:vortex-compact 265250976 / 412309710 / -35.7% 🟢 326971967 / 522290422 / -37.4% 🟢 0.81 / 0.79 / +2.8%
tpch_q18/datafusion:vortex-compact 349905402 / 631563876 / -44.6% 🟢 398639538 / 772562851 / -48.4% 🟢 0.88 / 0.82 / +7.4%
tpch_q19/datafusion:vortex-compact 87470021 / 152575866 / -42.7% 🟢 149266338 / 228668385 / -34.7% 🟢 0.59 / 0.67 / -12.2%
tpch_q20/datafusion:vortex-compact 87848994 / 138863389 / -36.7% 🟢 199060109 / 249619162 / -20.3% 🟢 0.44 / 0.56 / -20.7%
tpch_q21/datafusion:vortex-compact 288336294 / 460646447 / -37.4% 🟢 342004562 / 625576187 / -45.3% 🟢 0.84 / 0.74 / +14.5%
tpch_q22/datafusion:vortex-compact 29869867 / 42093538 / -29.0% 🟢 66024144 / 99719707 / -33.8% 🟢 0.45 / 0.42 / +7.2%
datafusion / parquet / ns (0.658x ✅, 22↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/datafusion:parquet 154814173 / 326461529 / -52.6% 🟢 251882278 / 414030288 / -39.2% 🟢 0.61 / 0.79 / -22.1%
tpch_q02/datafusion:parquet 139198712 / 183560540 / -24.2% 🟢 179296274 / 258596163 / -30.7% 🟢 0.78 / 0.71 / +9.4%
tpch_q03/datafusion:parquet 140019863 / 211963380 / -33.9% 🟢 306882161 / 359073418 / -14.5% 🟢 0.46 / 0.59 / -22.7%
tpch_q04/datafusion:parquet 62080628 / 88326579 / -29.7% 🟢 159819164 / 187397933 / -14.7% 🟢 0.39 / 0.47 / -17.6%
tpch_q05/datafusion:parquet 216493082 / 341626594 / -36.6% 🟢 379049786 / 532581904 / -28.8% 🟢 0.57 / 0.64 / -11.0%
tpch_q06/datafusion:parquet 41751089 / 67115126 / -37.8% 🟢 168395987 / 167484595 / +0.5% 0.25 / 0.40 / -38.1%
tpch_q07/datafusion:parquet 293340702 / 478589586 / -38.7% 🟢 448273772 / 665324734 / -32.6% 🟢 0.65 / 0.72 / -9.0%
tpch_q08/datafusion:parquet 250555037 / 376072323 / -33.4% 🟢 432026668 / 551320284 / -21.6% 🟢 0.58 / 0.68 / -15.0%
tpch_q09/datafusion:parquet 405059146 / 650990704 / -37.8% 🟢 581710658 / 830372052 / -29.9% 🟢 0.70 / 0.78 / -11.2%
tpch_q10/datafusion:parquet 379339035 / 511548710 / -25.8% 🟢 515547708 / 680366750 / -24.2% 🟢 0.74 / 0.75 / -2.1%
tpch_q11/datafusion:parquet 92934220 / 125179290 / -25.8% 🟢 129061134 / 206546483 / -37.5% 🟢 0.72 / 0.61 / +18.8%
tpch_q12/datafusion:parquet 85455931 / 138660178 / -38.4% 🟢 198152815 / 256880587 / -22.9% 🟢 0.43 / 0.54 / -20.1%
tpch_q13/datafusion:parquet 223243039 / 335585750 / -33.5% 🟢 308460445 / 415113721 / -25.7% 🟢 0.72 / 0.81 / -10.5%
tpch_q14/datafusion:parquet 75091104 / 114632925 / -34.5% 🟢 235796832 / 251672068 / -6.3% 0.32 / 0.46 / -30.1%
tpch_q15/datafusion:parquet 111575154 / 183794668 / -39.3% 🟢 237700150 / 288614240 / -17.6% 🟢 0.47 / 0.64 / -26.3%
tpch_q16/datafusion:parquet 91124594 / 123409100 / -26.2% 🟢 134492034 / 205413269 / -34.5% 🟢 0.68 / 0.60 / +12.8%
tpch_q17/datafusion:parquet 303939298 / 454784389 / -33.2% 🟢 419752600 / 573953330 / -26.9% 🟢 0.72 / 0.79 / -8.6%
tpch_q18/datafusion:parquet 382852438 / 693609076 / -44.8% 🟢 492800823 / 843930741 / -41.6% 🟢 0.78 / 0.82 / -5.5%
tpch_q19/datafusion:parquet 119251472 / 178724156 / -33.3% 🟢 275906103 / 309081204 / -10.7% 🟢 0.43 / 0.58 / -25.3%
tpch_q20/datafusion:parquet 184334393 / 273076576 / -32.5% 🟢 319742317 / 395053892 / -19.1% 🟢 0.58 / 0.69 / -16.6%
tpch_q21/datafusion:parquet 370896717 / 557858395 / -33.5% 🟢 513214230 / 789171872 / -35.0% 🟢 0.72 / 0.71 / +2.2%
tpch_q22/datafusion:parquet 183987528 / 223826157 / -17.8% 🟢 225831235 / 283724670 / -20.4% 🟢 0.81 / 0.79 / +3.3%
duckdb / vortex-file-compressed / ns (0.665x ✅, 22↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/duckdb:vortex-file-compressed 76481825 / 100550963 / -23.9% 🟢 156580548 / 201242670 / -22.2% 🟢 0.49 / 0.50 / -2.2%
tpch_q02/duckdb:vortex-file-compressed 38515234 / 65258498 / -41.0% 🟢 56880881 / 113432565 / -49.9% 🟢 0.68 / 0.58 / +17.7%
tpch_q03/duckdb:vortex-file-compressed 84323445 / 126916981 / -33.6% 🟢 152724994 / 243981892 / -37.4% 🟢 0.55 / 0.52 / +6.1%
tpch_q04/duckdb:vortex-file-compressed 107376328 / 161575120 / -33.5% 🟢 148445956 / 236583938 / -37.3% 🟢 0.72 / 0.68 / +5.9%
tpch_q05/duckdb:vortex-file-compressed 86501320 / 136828614 / -36.8% 🟢 163440038 / 257808419 / -36.6% 🟢 0.53 / 0.53 / -0.3%
tpch_q06/duckdb:vortex-file-compressed 31429390 / 35886663 / -12.4% 🟢 109252984 / 130783684 / -16.5% 🟢 0.29 / 0.27 / +4.8%
tpch_q07/duckdb:vortex-file-compressed 82700348 / 118894129 / -30.4% 🟢 191533726 / 284238115 / -32.6% 🟢 0.43 / 0.42 / +3.2%
tpch_q08/duckdb:vortex-file-compressed 108935983 / 169100385 / -35.6% 🟢 247486576 / 288765495 / -14.3% 🟢 0.44 / 0.59 / -24.8%
tpch_q09/duckdb:vortex-file-compressed 227978664 / 301484414 / -24.4% 🟢 321552168 / 436522842 / -26.3% 🟢 0.71 / 0.69 / +2.7%
tpch_q10/duckdb:vortex-file-compressed 143567967 / 211248541 / -32.0% 🟢 222819304 / 363150710 / -38.6% 🟢 0.64 / 0.58 / +10.8%
tpch_q11/duckdb:vortex-file-compressed 21492167 / 41835865 / -48.6% 🟢 39323240 / 95641261 / -58.9% 🟢 0.55 / 0.44 / +24.9%
tpch_q12/duckdb:vortex-file-compressed 70768921 / 111720986 / -36.7% 🟢 118075111 / 186959659 / -36.8% 🟢 0.60 / 0.60 / +0.3%
tpch_q13/duckdb:vortex-file-compressed 144679959 / 197956887 / -26.9% 🟢 176299470 / 290024576 / -39.2% 🟢 0.82 / 0.68 / +20.2%
tpch_q14/duckdb:vortex-file-compressed 39648260 / 54171283 / -26.8% 🟢 131891391 / 166763560 / -20.9% 🟢 0.30 / 0.32 / -7.5%
tpch_q15/duckdb:vortex-file-compressed 50648785 / 76733463 / -34.0% 🟢 134796454 / 198412182 / -32.1% 🟢 0.38 / 0.39 / -2.8%
tpch_q16/duckdb:vortex-file-compressed 53937018 / 83822054 / -35.7% 🟢 69254965 / 122035732 / -43.3% 🟢 0.78 / 0.69 / +13.4%
tpch_q17/duckdb:vortex-file-compressed 67089090 / 110304074 / -39.2% 🟢 156197687 / 292494845 / -46.6% 🟢 0.43 / 0.38 / +13.9%
tpch_q18/duckdb:vortex-file-compressed 180904782 / 268882900 / -32.7% 🟢 206086131 / 350007301 / -41.1% 🟢 0.88 / 0.77 / +14.3%
tpch_q19/duckdb:vortex-file-compressed 53629579 / 85584443 / -37.3% 🟢 180474093 / 219281937 / -17.7% 🟢 0.30 / 0.39 / -23.9%
tpch_q20/duckdb:vortex-file-compressed 79623603 / 128985826 / -38.3% 🟢 179890554 / 259462170 / -30.7% 🟢 0.44 / 0.50 / -11.0%
tpch_q21/duckdb:vortex-file-compressed 325821166 / 483170146 / -32.6% 🟢 378045618 / 566624118 / -33.3% 🟢 0.86 / 0.85 / +1.1%
tpch_q22/duckdb:vortex-file-compressed 46809250 / 74848896 / -37.5% 🟢 67299999 / 126794929 / -46.9% 🟢 0.70 / 0.59 / +17.8%
duckdb / vortex-compact / ns (0.699x ✅, 21↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/duckdb:vortex-compact 98754697 / 120119409 / -17.8% 🟢 153998598 / 203206324 / -24.2% 🟢 0.64 / 0.59 / +8.5%
tpch_q02/duckdb:vortex-compact 40988189 / 71889454 / -43.0% 🟢 55888326 / 115592276 / -51.7% 🟢 0.73 / 0.62 / +17.9%
tpch_q03/duckdb:vortex-compact 85213123 / 117649699 / -27.6% 🟢 145948982 / 224638507 / -35.0% 🟢 0.58 / 0.52 / +11.5%
tpch_q04/duckdb:vortex-compact 117274420 / 160588342 / -27.0% 🟢 142873485 / 245280198 / -41.8% 🟢 0.82 / 0.65 / +25.4%
tpch_q05/duckdb:vortex-compact 94074761 / 148330838 / -36.6% 🟢 161118246 / 273499250 / -41.1% 🟢 0.58 / 0.54 / +7.7%
tpch_q06/duckdb:vortex-compact 37983983 / 41161741 / -7.7% 97168153 / 133503089 / -27.2% 🟢 0.39 / 0.31 / +26.8%
tpch_q07/duckdb:vortex-compact 97867697 / 135621260 / -27.8% 🟢 184389185 / 266929131 / -30.9% 🟢 0.53 / 0.51 / +4.5%
tpch_q08/duckdb:vortex-compact 117588265 / 181885785 / -35.4% 🟢 211439301 / 307359372 / -31.2% 🟢 0.56 / 0.59 / -6.0%
tpch_q09/duckdb:vortex-compact 236523704 / 315659193 / -25.1% 🟢 321667189 / 446966702 / -28.0% 🟢 0.74 / 0.71 / +4.1%
tpch_q10/duckdb:vortex-compact 152252169 / 213907314 / -28.8% 🟢 212266335 / 344229765 / -38.3% 🟢 0.72 / 0.62 / +15.4%
tpch_q11/duckdb:vortex-compact 23495665 / 41618886 / -43.5% 🟢 41448699 / 121116409 / -65.8% 🟢 0.57 / 0.34 / +65.0%
tpch_q12/duckdb:vortex-compact 96967002 / 119679647 / -19.0% 🟢 126951545 / 207535161 / -38.8% 🟢 0.76 / 0.58 / +32.5%
tpch_q13/duckdb:vortex-compact 155935425 / 201844006 / -22.7% 🟢 181864963 / 315729281 / -42.4% 🟢 0.86 / 0.64 / +34.1%
tpch_q14/duckdb:vortex-compact 47731618 / 58912327 / -19.0% 🟢 125646038 / 163492185 / -23.1% 🟢 0.38 / 0.36 / +5.4%
tpch_q15/duckdb:vortex-compact 63591598 / 81351463 / -21.8% 🟢 145929191 / 211496062 / -31.0% 🟢 0.44 / 0.38 / +13.3%
tpch_q16/duckdb:vortex-compact 55181217 / 90326972 / -38.9% 🟢 69634725 / 127830307 / -45.5% 🟢 0.79 / 0.71 / +12.1%
tpch_q17/duckdb:vortex-compact 69445525 / 118194567 / -41.2% 🟢 139294868 / 242164236 / -42.5% 🟢 0.50 / 0.49 / +2.1%
tpch_q18/duckdb:vortex-compact 179537958 / 266649639 / -32.7% 🟢 202293366 / 356002937 / -43.2% 🟢 0.89 / 0.75 / +18.5%
tpch_q19/duckdb:vortex-compact 62275777 / 92340526 / -32.6% 🟢 150837101 / 204905717 / -26.4% 🟢 0.41 / 0.45 / -8.4%
tpch_q20/duckdb:vortex-compact 89487655 / 148244569 / -39.6% 🟢 173590072 / 270807646 / -35.9% 🟢 0.52 / 0.55 / -5.8%
tpch_q21/duckdb:vortex-compact 355876094 / 476022445 / -25.2% 🟢 396415566 / 552794608 / -28.3% 🟢 0.90 / 0.86 / +4.3%
tpch_q22/duckdb:vortex-compact 48855179 / 77185178 / -36.7% 🟢 68377198 / 151545832 / -54.9% 🟢 0.71 / 0.51 / +40.3%
duckdb / parquet / ns (0.874x ✅, 12↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/duckdb:parquet 90628974 / 143778896 / -37.0% 🟢 129457724 / 195289430 / -33.7% 🟢 0.70 / 0.74 / -4.9%
tpch_q02/duckdb:parquet 91241537 / 101652548 / -10.2% 🟢 107182087 / 121018731 / -11.4% 🟢 0.85 / 0.84 / +1.3%
tpch_q03/duckdb:parquet 157476541 / 165614077 / -4.9% 242317253 / 232842071 / +4.1% 0.65 / 0.71 / -8.6%
tpch_q04/duckdb:parquet 98905164 / 119235027 / -17.1% 🟢 148725490 / 154661012 / -3.8% 0.67 / 0.77 / -13.7%
tpch_q05/duckdb:parquet 168705411 / 187330844 / -9.9% 257575949 / 261569193 / -1.5% 0.65 / 0.72 / -8.5%
tpch_q06/duckdb:parquet 46907192 / 58230902 / -19.4% 🟢 107349435 / 94424939 / +13.7% 🔴 0.44 / 0.62 / -29.1%
tpch_q07/duckdb:parquet 130642697 / 145762074 / -10.4% 🟢 223745670 / 217714004 / +2.8% 0.58 / 0.67 / -12.8%
tpch_q08/duckdb:parquet 197218206 / 217411172 / -9.3% 348704382 / 305353729 / +14.2% 🔴 0.57 / 0.71 / -20.6%
tpch_q09/duckdb:parquet 292365916 / 347280701 / -15.8% 🟢 384341471 / 430041874 / -10.6% 🟢 0.76 / 0.81 / -5.8%
tpch_q10/duckdb:parquet 602700308 / 656398683 / -8.2% 687901201 / 730228597 / -5.8% 0.88 / 0.90 / -2.5%
tpch_q11/duckdb:parquet 41203143 / 41792832 / -1.4% 51859085 / 54042992 / -4.0% 0.79 / 0.77 / +2.7%
tpch_q12/duckdb:parquet 100517560 / 121855529 / -17.5% 🟢 150995869 / 150043233 / +0.6% 0.67 / 0.81 / -18.0%
tpch_q13/duckdb:parquet 408347634 / 398462559 / +2.5% 419395478 / 424360483 / -1.2% 0.97 / 0.94 / +3.7%
tpch_q14/duckdb:parquet 139969589 / 150336862 / -6.9% 235805871 / 216882420 / +8.7% 0.59 / 0.69 / -14.4%
tpch_q15/duckdb:parquet 64461192 / 76213890 / -15.4% 🟢 136974627 / 121749054 / +12.5% 🔴 0.47 / 0.63 / -24.8%
tpch_q16/duckdb:parquet 156590763 / 154867083 / +1.1% 168713578 / 203059286 / -16.9% 🟢 0.93 / 0.76 / +21.7%
tpch_q17/duckdb:parquet 93074033 / 127341468 / -26.9% 🟢 190546638 / 209657957 / -9.1% 0.49 / 0.61 / -19.6%
tpch_q18/duckdb:parquet 228963725 / 279470745 / -18.1% 🟢 221003856 / 292165745 / -24.4% 🟢 1.04 / 0.96 / +8.3%
tpch_q19/duckdb:parquet 221660618 / 229328870 / -3.3% 307735500 / 321329624 / -4.2% 0.72 / 0.71 / +0.9%
tpch_q20/duckdb:parquet 171517850 / 196604143 / -12.8% 🟢 252013270 / 271578991 / -7.2% 0.68 / 0.72 / -6.0%
tpch_q21/duckdb:parquet 349670462 / 441743635 / -20.8% 🟢 381202744 / 481539821 / -20.8% 🟢 0.92 / 0.92 / -0.0%
tpch_q22/duckdb:parquet 321317787 / 338324755 / -5.0% 335054503 / 356224009 / -5.9% 0.96 / 0.95 / +1.0%
File Size Changes (5 files changed, +0.0% overall, 4↑ 1↓)
File Scale Format Base HEAD Change %
orders.vortex 10.0 vortex-file-compressed 390.32 MB 390.34 MB +25.05 KB +0.0%
supplier.vortex 10.0 vortex-file-compressed 5.40 MB 5.40 MB +80 B +0.0%
lineitem.vortex 10.0 vortex-file-compressed 1.56 GB 1.56 GB +14.02 KB +0.0%
customer.vortex 10.0 vortex-file-compressed 82.86 MB 82.86 MB +408 B +0.0%
part.vortex 10.0 vortex-file-compressed 48.42 MB 48.41 MB 5.84 KB -0.0%

Totals:

  • vortex-compact: 1.92 GB → 1.92 GB (0.0%)
  • vortex-file-compressed: 2.31 GB → 2.31 GB (+0.0%)

@github-actions

github-actions Bot commented Sep 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmarks: Statistical and Population Genetics 📖

Commits: PR dd8b5948 vs base 792cef74
Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: -14.1%
Engines: DuckDB No clear signal (-14.1%, environment too noisy confidence)
Vortex (geomean): hot 0.754x ✅ · cold 0.742x ✅
Parquet (geomean): hot 0.878x ✅ · cold 0.830x ✅
Shifts: Parquet (control) -12.2% · Median polish -24.6%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Table cells: Each cell reads PR / base / %diff. The hot and cold columns mark a change as 🔴 slower or 🟢 faster once it clears this suite's threshold; anything smaller is noise here and is left unmarked, as is hot/cold, because a shift in the warm-up ratio is not a win or a loss by itself.
duckdb / vortex-file-compressed / ns (0.747x ✅, 9↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
statpopgen_q00/duckdb:vortex-file-compressed 11365231 / 12633670 / -10.0% 🟢 11955411 / 13584630 / -12.0% 🟢 0.95 / 0.93 / +2.2%
statpopgen_q01/duckdb:vortex-file-compressed 13060220 / 13831510 / -5.6% 18726596 / 24557702 / -23.7% 🟢 0.70 / 0.56 / +23.8%
statpopgen_q02/duckdb:vortex-file-compressed 456769372 / 611214797 / -25.3% 🟢 481090196 / 632353033 / -23.9% 🟢 0.95 / 0.97 / -1.8%
statpopgen_q03/duckdb:vortex-file-compressed 1013268820 / 1321320534 / -23.3% 🟢 1061671338 / 1308559215 / -18.9% 🟢 0.95 / 1.01 / -5.5%
statpopgen_q04/duckdb:vortex-file-compressed 1018452212 / 1326567513 / -23.2% 🟢 1029795151 / 1290715752 / -20.2% 🟢 0.99 / 1.03 / -3.8%
statpopgen_q05/duckdb:vortex-file-compressed 477643417 / 903920840 / -47.2% 🟢 501803415 / 661275147 / -24.1% 🟢 0.95 / 1.37 / -30.4%
statpopgen_q06/duckdb:vortex-file-compressed 1315037412 / 2265974002 / -42.0% 🟢 1325014128 / 1793048265 / -26.1% 🟢 0.99 / 1.26 / -21.5%
statpopgen_q07/duckdb:vortex-file-compressed 217250488 / 240813611 / -9.8% 233959555 / 351145112 / -33.4% 🟢 0.93 / 0.69 / +35.4%
statpopgen_q08/duckdb:vortex-file-compressed 238244758 / 267489144 / -10.9% 🟢 265468355 / 310590521 / -14.5% 🟢 0.90 / 0.86 / +4.2%
statpopgen_q09/duckdb:vortex-file-compressed 807080666 / 1284623354 / -37.2% 🟢 859909908 / 1072451009 / -19.8% 🟢 0.94 / 1.20 / -21.6%
statpopgen_q10/duckdb:vortex-file-compressed 2239287520 / 3223961742 / -30.5% 🟢 2226952960 / 3194811928 / -30.3% 🟢 1.01 / 1.01 / -0.4%
duckdb / vortex-compact / ns (0.762x ✅, 9↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
statpopgen_q00/duckdb:vortex-compact 11314454 / 11981802 / -5.6% 12106581 / 14470542 / -16.3% 🟢 0.93 / 0.83 / +12.9%
statpopgen_q01/duckdb:vortex-compact 12719942 / 13427852 / -5.3% 17967133 / 22953076 / -21.7% 🟢 0.71 / 0.59 / +21.0%
statpopgen_q02/duckdb:vortex-compact 476998348 / 670095538 / -28.8% 🟢 487436891 / 691531307 / -29.5% 🟢 0.98 / 0.97 / +1.0%
statpopgen_q03/duckdb:vortex-compact 1043160610 / 1408947648 / -26.0% 🟢 1042048684 / 1815387704 / -42.6% 🟢 1.00 / 0.78 / +29.0%
statpopgen_q04/duckdb:vortex-compact 1034888986 / 1396770562 / -25.9% 🟢 1048719462 / 1484961834 / -29.4% 🟢 0.99 / 0.94 / +4.9%
statpopgen_q05/duckdb:vortex-compact 496769870 / 692409665 / -28.3% 🟢 504258936 / 696673700 / -27.6% 🟢 0.99 / 0.99 / -0.9%
statpopgen_q06/duckdb:vortex-compact 1331332035 / 1816388634 / -26.7% 🟢 1332416591 / 1821449710 / -26.8% 🟢 1.00 / 1.00 / +0.2%
statpopgen_q07/duckdb:vortex-compact 375855078 / 498610188 / -24.6% 🟢 396542708 / 559495052 / -29.1% 🟢 0.95 / 0.89 / +6.4%
statpopgen_q08/duckdb:vortex-compact 392336586 / 513243392 / -23.6% 🟢 416759627 / 563065637 / -26.0% 🟢 0.94 / 0.91 / +3.3%
statpopgen_q09/duckdb:vortex-compact 818289307 / 1224288419 / -33.2% 🟢 848247021 / 1376344629 / -38.4% 🟢 0.96 / 0.89 / +8.4%
statpopgen_q10/duckdb:vortex-compact 2271259100 / 3183553964 / -28.7% 🟢 2245690875 / 2987184092 / -24.8% 🟢 1.01 / 1.07 / -5.1%
duckdb / parquet / ns (0.878x ✅, 7↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
statpopgen_q00/duckdb:parquet 341402060 / 376852060 / -9.4% 345327777 / 380187540 / -9.2% 0.99 / 0.99 / -0.3%
statpopgen_q01/duckdb:parquet 413364049 / 466266268 / -11.3% 🟢 424347305 / 478974976 / -11.4% 🟢 0.97 / 0.97 / +0.1%
statpopgen_q02/duckdb:parquet 624105073 / 769573821 / -18.9% 🟢 645128342 / 804210172 / -19.8% 🟢 0.97 / 0.96 / +1.1%
statpopgen_q03/duckdb:parquet 998863708 / 1058045237 / -5.6% 924865739 / 1130328104 / -18.2% 🟢 1.08 / 0.94 / +15.4%
statpopgen_q04/duckdb:parquet 947238008 / 1056611624 / -10.4% 🟢 911358797 / 1117460660 / -18.4% 🟢 1.04 / 0.95 / +9.9%
statpopgen_q05/duckdb:parquet 729214185 / 769961702 / -5.3% 667886219 / 813212856 / -17.9% 🟢 1.09 / 0.95 / +15.3%
statpopgen_q06/duckdb:parquet 928061778 / 1023296180 / -9.3% 853362746 / 1038595178 / -17.8% 🟢 1.09 / 0.99 / +10.4%
statpopgen_q07/duckdb:parquet 813828874 / 937687145 / -13.2% 🟢 907662831 / 1099687130 / -17.5% 🟢 0.90 / 0.85 / +5.2%
statpopgen_q08/duckdb:parquet 829012770 / 972829454 / -14.8% 🟢 924072766 / 1148747214 / -19.6% 🟢 0.90 / 0.85 / +5.9%
statpopgen_q09/duckdb:parquet 814994352 / 993741858 / -18.0% 🟢 836658304 / 1031524769 / -18.9% 🟢 0.97 / 0.96 / +1.1%
statpopgen_q10/duckdb:parquet 1370444152 / 1647790020 / -16.8% 🟢 1370917075 / 1675730296 / -18.2% 🟢 1.00 / 0.98 / +1.7%
File Size Changes (1 files changed, -0.0% overall, 0↑ 1↓)
File Scale Format Base HEAD Change %
gnomad.genomes.v3.1.2.hgdp_tgp.chr21.vortex 100000 vortex-file-compressed 1.55 GB 1.55 GB 8.03 KB -0.0%

Totals:

  • vortex-compact: 960.83 MB → 960.83 MB (0.0%)
  • vortex-file-compressed: 1.55 GB → 1.55 GB (-0.0%)

@github-actions

github-actions Bot commented Sep 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-H SF=10 on S3 📖

Commits: PR dd8b5948 vs base 792cef74
Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: +18.7%
Engines: DataFusion No clear signal (-0.4%, environment too noisy confidence) · DuckDB Likely regression (+41.4%, environment too noisy confidence)
Vortex (geomean): hot 1.207x ➖ · cold 1.154x ➖
Parquet (geomean): hot 1.017x ➖ · cold 1.016x ➖
Shifts: Parquet (control) +1.7% · Median polish -0.3%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Table cells: Each cell reads PR / base / %diff. The hot and cold columns mark a change as 🔴 slower or 🟢 faster once it clears this suite's threshold; anything smaller is noise here and is left unmarked, as is hot/cold, because a shift in the warm-up ratio is not a win or a loss by itself.
datafusion / vortex-compact / ns (1.006x ➖, 2↑ 4↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/datafusion:vortex-compact 374176161 / 590685818 / -36.7% 🟢 830560125 / 1022583756 / -18.8% 0.45 / 0.58 / -22.0%
tpch_q02/datafusion:vortex-compact 561765142 / 467683043 / +20.1% 1504059797 / 1105735975 / +36.0% 🔴 0.37 / 0.42 / -11.7%
tpch_q03/datafusion:vortex-compact 789089580 / 455367927 / +73.3% 🔴 1426893932 / 1307754497 / +9.1% 0.55 / 0.35 / +58.8%
tpch_q04/datafusion:vortex-compact 374333408 / 265822765 / +40.8% 🔴 590293949 / 626381643 / -5.8% 0.63 / 0.42 / +49.4%
tpch_q05/datafusion:vortex-compact 694818870 / 1339903820 / -48.1% 🟢 1065319280 / 1648764407 / -35.4% 🟢 0.65 / 0.81 / -19.7%
tpch_q06/datafusion:vortex-compact 313030062 / 287007973 / +9.1% 501004279 / 516228370 / -2.9% 0.62 / 0.56 / +12.4%
tpch_q07/datafusion:vortex-compact 672060410 / 804322729 / -16.4% 807949779 / 1000919345 / -19.3% 0.83 / 0.80 / +3.5%
tpch_q08/datafusion:vortex-compact 792542360 / 825369641 / -4.0% 1097320121 / 1054816016 / +4.0% 0.72 / 0.78 / -7.7%
tpch_q09/datafusion:vortex-compact 818294190 / 900504225 / -9.1% 2081792572 / 1492872447 / +39.4% 🔴 0.39 / 0.60 / -34.8%
tpch_q10/datafusion:vortex-compact 557518350 / 656901207 / -15.1% 950837782 / 816987042 / +16.4% 0.59 / 0.80 / -27.1%
tpch_q11/datafusion:vortex-compact 354286945 / 337936427 / +4.8% 636758820 / 582836362 / +9.3% 0.56 / 0.58 / -4.0%
tpch_q12/datafusion:vortex-compact 762571531 / 539452883 / +41.4% 🔴 1036376310 / 1445489379 / -28.3% 0.74 / 0.37 / +97.2%
tpch_q13/datafusion:vortex-compact 293101333 / 282690638 / +3.7% 707843385 / 713398765 / -0.8% 0.41 / 0.40 / +4.5%
tpch_q14/datafusion:vortex-compact 413408405 / 351352568 / +17.7% 587828665 / 452568262 / +29.9% 0.70 / 0.78 / -9.4%
tpch_q15/datafusion:vortex-compact 817087538 / 533309528 / +53.2% 🔴 1018284060 / 1067851388 / -4.6% 0.80 / 0.50 / +60.7%
tpch_q16/datafusion:vortex-compact 250471975 / 269951100 / -7.2% 425764937 / 491722436 / -13.4% 0.59 / 0.55 / +7.2%
tpch_q17/datafusion:vortex-compact 807204819 / 736247079 / +9.6% 1040305903 / 870697634 / +19.5% 0.78 / 0.85 / -8.2%
tpch_q18/datafusion:vortex-compact 642987092 / 813101187 / -20.9% 762745278 / 1121715695 / -32.0% 🟢 0.84 / 0.72 / +16.3%
tpch_q19/datafusion:vortex-compact 858914860 / 720535547 / +19.2% 799730329 / 913373386 / -12.4% 1.07 / 0.79 / +36.1%
tpch_q20/datafusion:vortex-compact 676506897 / 855668340 / -20.9% 1042684070 / 931138120 / +12.0% 0.65 / 0.92 / -29.4%
tpch_q21/datafusion:vortex-compact 748159531 / 924408478 / -19.1% 1120370905 / 1017216495 / +10.1% 0.67 / 0.91 / -26.5%
tpch_q22/datafusion:vortex-compact 308550724 / 296986177 / +3.9% 736825604 / 962336641 / -23.4% 0.42 / 0.31 / +35.7%
datafusion / parquet / ns (1.010x ➖, 1↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/datafusion:parquet 367977924 / 418549709 / -12.1% 759348121 / 819735536 / -7.4% 0.48 / 0.51 / -5.1%
tpch_q02/datafusion:parquet 524750942 / 570484092 / -8.0% 940971320 / 1049092837 / -10.3% 0.56 / 0.54 / +2.6%
tpch_q03/datafusion:parquet 821802959 / 634673139 / +29.5% 1530208300 / 1410942791 / +8.5% 0.54 / 0.45 / +19.4%
tpch_q04/datafusion:parquet 409744579 / 401654266 / +2.0% 507991108 / 570043978 / -10.9% 0.81 / 0.70 / +14.5%
tpch_q05/datafusion:parquet 677877991 / 702936307 / -3.6% 973739238 / 976563821 / -0.3% 0.70 / 0.72 / -3.3%
tpch_q06/datafusion:parquet 234802038 / 193078124 / +21.6% 412729840 / 372144113 / +10.9% 0.57 / 0.52 / +9.7%
tpch_q07/datafusion:parquet 753491145 / 845529245 / -10.9% 1657954443 / 1007709393 / +64.5% 🔴 0.45 / 0.84 / -45.8%
tpch_q08/datafusion:parquet 1143860581 / 888332674 / +28.8% 1549057180 / 1150545804 / +34.6% 🔴 0.74 / 0.77 / -4.4%
tpch_q09/datafusion:parquet 1027761405 / 1489441626 / -31.0% 🟢 1501328781 / 1524833557 / -1.5% 0.68 / 0.98 / -29.9%
tpch_q10/datafusion:parquet 1675414789 / 1691184231 / -0.9% 2443745223 / 2413074721 / +1.3% 0.69 / 0.70 / -2.2%
tpch_q11/datafusion:parquet 406978233 / 408329337 / -0.3% 543614719 / 601365796 / -9.6% 0.75 / 0.68 / +10.3%
tpch_q12/datafusion:parquet 466392848 / 433264305 / +7.6% 817737392 / 589199893 / +38.8% 🔴 0.57 / 0.74 / -22.4%
tpch_q13/datafusion:parquet 493618456 / 620140413 / -20.4% 843209173 / 1012323155 / -16.7% 0.59 / 0.61 / -4.4%
tpch_q14/datafusion:parquet 426181468 / 421448089 / +1.1% 741850852 / 598671828 / +23.9% 0.57 / 0.70 / -18.4%
tpch_q15/datafusion:parquet 875300629 / 685417329 / +27.7% 1152782909 / 954897839 / +20.7% 0.76 / 0.72 / +5.8%
tpch_q16/datafusion:parquet 276373727 / 279918605 / -1.3% 460242178 / 526791845 / -12.6% 0.60 / 0.53 / +13.0%
tpch_q17/datafusion:parquet 876501182 / 781135163 / +12.2% 946607999 / 862188156 / +9.8% 0.93 / 0.91 / +2.2%
tpch_q18/datafusion:parquet 1119205983 / 1135253700 / -1.4% 1258571707 / 1459739012 / -13.8% 0.89 / 0.78 / +14.3%
tpch_q19/datafusion:parquet 486418118 / 468379085 / +3.9% 710323239 / 601341002 / +18.1% 0.68 / 0.78 / -12.1%
tpch_q20/datafusion:parquet 785686866 / 762207816 / +3.1% 909964361 / 1027610651 / -11.4% 0.86 / 0.74 / +16.4%
tpch_q21/datafusion:parquet 1213551394 / 1152568048 / +5.3% 1250276537 / 1375265033 / -9.1% 0.97 / 0.84 / +15.8%
tpch_q22/datafusion:parquet 621191411 / 665063927 / -6.6% 900676843 / 970857015 / -7.2% 0.69 / 0.69 / +0.7%
duckdb / vortex-compact / ns (1.449x ❌, 0↑ 17↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/duckdb:vortex-compact 1318549039 / 777101994 / +69.7% 🔴 3435045977 / 1864533774 / +84.2% 🔴 0.38 / 0.42 / -7.9%
tpch_q02/duckdb:vortex-compact 503269439 / 546499118 / -7.9% 773244876 / 877204537 / -11.9% 0.65 / 0.62 / +4.5%
tpch_q03/duckdb:vortex-compact 1517827517 / 933674916 / +62.6% 🔴 1815109737 / 1670575218 / +8.7% 0.84 / 0.56 / +49.6%
tpch_q04/duckdb:vortex-compact 918566443 / 694655956 / +32.2% 🔴 1056689580 / 889511633 / +18.8% 0.87 / 0.78 / +11.3%
tpch_q05/duckdb:vortex-compact 1445586029 / 863234208 / +67.5% 🔴 1666306016 / 1143790234 / +45.7% 🔴 0.87 / 0.75 / +14.9%
tpch_q06/duckdb:vortex-compact 1579724987 / 861800585 / +83.3% 🔴 1594708982 / 885114937 / +80.2% 🔴 0.99 / 0.97 / +1.7%
tpch_q07/duckdb:vortex-compact 1850786362 / 1170769209 / +58.1% 🔴 1917132107 / 1562939301 / +22.7% 0.97 / 0.75 / +28.9%
tpch_q08/duckdb:vortex-compact 1761072266 / 1209232645 / +45.6% 🔴 2767948517 / 1470409517 / +88.2% 🔴 0.64 / 0.82 / -22.6%
tpch_q09/duckdb:vortex-compact 2365719534 / 1607206797 / +47.2% 🔴 2609098653 / 1712668900 / +52.3% 🔴 0.91 / 0.94 / -3.4%
tpch_q10/duckdb:vortex-compact 1626560976 / 1181446569 / +37.7% 🔴 2096837526 / 1868819866 / +12.2% 0.78 / 0.63 / +22.7%
tpch_q11/duckdb:vortex-compact 489037507 / 461992713 / +5.9% 701557717 / 663118165 / +5.8% 0.70 / 0.70 / +0.1%
tpch_q12/duckdb:vortex-compact 2587051945 / 1423355768 / +81.8% 🔴 2932664256 / 1736976214 / +68.8% 🔴 0.88 / 0.82 / +7.7%
tpch_q13/duckdb:vortex-compact 1336656566 / 858690495 / +55.7% 🔴 2441599479 / 1288319582 / +89.5% 🔴 0.55 / 0.67 / -17.9%
tpch_q14/duckdb:vortex-compact 1247206240 / 671345069 / +85.8% 🔴 1202947216 / 751529179 / +60.1% 🔴 1.04 / 0.89 / +16.1%
tpch_q15/duckdb:vortex-compact 1195530453 / 666682145 / +79.3% 🔴 1276172017 / 748263601 / +70.6% 🔴 0.94 / 0.89 / +5.1%
tpch_q16/duckdb:vortex-compact 232976143 / 263849420 / -11.7% 273363090 / 337185122 / -18.9% 0.85 / 0.78 / +8.9%
tpch_q17/duckdb:vortex-compact 1704997117 / 1102147990 / +54.7% 🔴 1724273490 / 1163208678 / +48.2% 🔴 0.99 / 0.95 / +4.4%
tpch_q18/duckdb:vortex-compact 1251327378 / 1045740067 / +19.7% 1252529607 / 1074142301 / +16.6% 1.00 / 0.97 / +2.6%
tpch_q19/duckdb:vortex-compact 1421689161 / 1144216560 / +24.3% 1763299840 / 1282251588 / +37.5% 🔴 0.81 / 0.89 / -9.6%
tpch_q20/duckdb:vortex-compact 2145806793 / 1218497839 / +76.1% 🔴 2189046801 / 1479072603 / +48.0% 🔴 0.98 / 0.82 / +19.0%
tpch_q21/duckdb:vortex-compact 3039127960 / 1988902924 / +52.8% 🔴 3011915275 / 1981687518 / +52.0% 🔴 1.01 / 1.00 / +0.5%
tpch_q22/duckdb:vortex-compact 352483303 / 265084496 / +33.0% 🔴 561576008 / 481190319 / +16.7% 0.63 / 0.55 / +13.9%
duckdb / parquet / ns (1.025x ➖, 0↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
tpch_q01/duckdb:parquet 392379481 / 383760332 / +2.2% 560488937 / 466626202 / +20.1% 0.70 / 0.82 / -14.9%
tpch_q02/duckdb:parquet 795672679 / 793592942 / +0.3% 1184658601 / 1336302124 / -11.3% 0.67 / 0.59 / +13.1%
tpch_q03/duckdb:parquet 1123692085 / 947395100 / +18.6% 1603807062 / 1419382710 / +13.0% 0.70 / 0.67 / +5.0%
tpch_q04/duckdb:parquet 599087734 / 653020818 / -8.3% 632582175 / 834201894 / -24.2% 0.95 / 0.78 / +21.0%
tpch_q05/duckdb:parquet 1233537945 / 1169252609 / +5.5% 1629137631 / 1165416496 / +39.8% 🔴 0.76 / 1.00 / -24.5%
tpch_q06/duckdb:parquet 375238848 / 434721346 / -13.7% 374205482 / 533280520 / -29.8% 1.00 / 0.82 / +23.0%
tpch_q07/duckdb:parquet 1282804409 / 995091306 / +28.9% 1655076504 / 959983616 / +72.4% 🔴 0.78 / 1.04 / -25.2%
tpch_q08/duckdb:parquet 1403908139 / 1403178664 / +0.1% 1674116698 / 1556755653 / +7.5% 0.84 / 0.90 / -7.0%
tpch_q09/duckdb:parquet 1432410379 / 1269393962 / +12.8% 1702062752 / 1545574875 / +10.1% 0.84 / 0.82 / +2.5%
tpch_q10/duckdb:parquet 2695898189 / 2447638163 / +10.1% 2883788202 / 2865087443 / +0.7% 0.93 / 0.85 / +9.4%
tpch_q11/duckdb:parquet 455196370 / 496830194 / -8.4% 599040492 / 717811691 / -16.5% 0.76 / 0.69 / +9.8%
tpch_q12/duckdb:parquet 703710683 / 691450253 / +1.8% 1027211614 / 1102287573 / -6.8% 0.69 / 0.63 / +9.2%
tpch_q13/duckdb:parquet 1037621799 / 962342282 / +7.8% 1297189869 / 1191195583 / +8.9% 0.80 / 0.81 / -1.0%
tpch_q14/duckdb:parquet 769036074 / 735950972 / +4.5% 689071898 / 700489267 / -1.6% 1.12 / 1.05 / +6.2%
tpch_q15/duckdb:parquet 539220866 / 449503389 / +20.0% 520584710 / 561150634 / -7.2% 1.04 / 0.80 / +29.3%
tpch_q16/duckdb:parquet 697331642 / 691173837 / +0.9% 781854061 / 704202204 / +11.0% 0.89 / 0.98 / -9.1%
tpch_q17/duckdb:parquet 639488740 / 624067427 / +2.5% 667845917 / 812849461 / -17.8% 0.96 / 0.77 / +24.7%
tpch_q18/duckdb:parquet 845566953 / 890866091 / -5.1% 1088750539 / 1040833080 / +4.6% 0.78 / 0.86 / -9.3%
tpch_q19/duckdb:parquet 855208435 / 925086538 / -7.6% 951899552 / 1048876607 / -9.2% 0.90 / 0.88 / +1.9%
tpch_q20/duckdb:parquet 1193109669 / 1217836942 / -2.0% 1383766806 / 1395650366 / -0.9% 0.86 / 0.87 / -1.2%
tpch_q21/duckdb:parquet 1040668875 / 1040853139 / -0.0% 867380227 / 1044067949 / -16.9% 1.20 / 1.00 / +20.3%
tpch_q22/duckdb:parquet 838163814 / 896452853 / -6.5% 885346350 / 987858760 / -10.4% 0.95 / 0.91 / +4.3%

@github-actions

github-actions Bot commented Sep 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmarks: Appian on NVME 📖

Commits: PR dd8b5948 vs base 792cef74
Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: -4.2%
Engines: DataFusion No clear signal (-0.7%, environment too noisy confidence) · DuckDB No clear signal (-7.6%, environment too noisy confidence)
Vortex (geomean): hot 0.639x ✅ · cold 0.575x ✅
Parquet (geomean): hot 0.667x ✅ · cold 0.646x ✅
Shifts: Parquet (control) -33.3% · Median polish -28.7%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Table cells: Each cell reads PR / base / %diff. The hot and cold columns mark a change as 🔴 slower or 🟢 faster once it clears this suite's threshold; anything smaller is noise here and is left unmarked, as is hot/cold, because a shift in the warm-up ratio is not a win or a loss by itself.
datafusion / vortex-compact / ns (0.686x ✅, 8↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
appian_q01/datafusion:vortex-compact 64194111 / 95021763 / -32.4% 🟢 95002801 / 157180690 / -39.6% 🟢 0.68 / 0.60 / +11.8%
appian_q02/datafusion:vortex-compact 298652803 / 446903663 / -33.2% 🟢 337759265 / 560415848 / -39.7% 🟢 0.88 / 0.80 / +10.9%
appian_q03/datafusion:vortex-compact 180587550 / 291751214 / -38.1% 🟢 232474704 / 419457700 / -44.6% 🟢 0.78 / 0.70 / +11.7%
appian_q04/datafusion:vortex-compact 11259605117 / 13559864790 / -17.0% 🟢 11023572606 / 13810954464 / -20.2% 🟢 1.02 / 0.98 / +4.0%
appian_q05/datafusion:vortex-compact 142667588 / 205901538 / -30.7% 🟢 179312187 / 287658224 / -37.7% 🟢 0.80 / 0.72 / +11.2%
appian_q06/datafusion:vortex-compact 178776880 / 265243578 / -32.6% 🟢 206913690 / 329944435 / -37.3% 🟢 0.86 / 0.80 / +7.5%
appian_q07/datafusion:vortex-compact 196535287 / 287214451 / -31.6% 🟢 243370236 / 402104935 / -39.5% 🟢 0.81 / 0.71 / +13.1%
appian_q08/datafusion:vortex-compact 829335714 / 1259895489 / -34.2% 🟢 819647397 / 1392960321 / -41.2% 🟢 1.01 / 0.90 / +11.9%
datafusion / parquet / ns (0.691x ✅, 8↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
appian_q01/datafusion:parquet 75542575 / 109129160 / -30.8% 🟢 108005485 / 172109832 / -37.2% 🟢 0.70 / 0.63 / +10.3%
appian_q02/datafusion:parquet 334175295 / 494935488 / -32.5% 🟢 385638812 / 596941239 / -35.4% 🟢 0.87 / 0.83 / +4.5%
appian_q03/datafusion:parquet 203459131 / 310686274 / -34.5% 🟢 256631711 / 458328542 / -44.0% 🟢 0.79 / 0.68 / +17.0%
appian_q04/datafusion:parquet 11328166169 / 13732428401 / -17.5% 🟢 11080721259 / 13878362809 / -20.2% 🟢 1.02 / 0.99 / +3.3%
appian_q05/datafusion:parquet 178467944 / 250155688 / -28.7% 🟢 226055674 / 327535255 / -31.0% 🟢 0.79 / 0.76 / +3.4%
appian_q06/datafusion:parquet 199080028 / 297129121 / -33.0% 🟢 235823057 / 382590780 / -38.4% 🟢 0.84 / 0.78 / +8.7%
appian_q07/datafusion:parquet 223885448 / 326595677 / -31.4% 🟢 267659633 / 432578766 / -38.1% 🟢 0.84 / 0.75 / +10.8%
appian_q08/datafusion:parquet 833214717 / 1331505448 / -37.4% 🟢 861598700 / 1447791323 / -40.5% 🟢 0.97 / 0.92 / +5.2%
duckdb / vortex-compact / ns (0.595x ✅, 8↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
appian_q01/duckdb:vortex-compact 123562553 / 152275442 / -18.9% 🟢 138799067 / 206444443 / -32.8% 🟢 0.89 / 0.74 / +20.7%
appian_q02/duckdb:vortex-compact 345062659 / 408874458 / -15.6% 🟢 357118342 / 469978927 / -24.0% 🟢 0.97 / 0.87 / +11.1%
appian_q03/duckdb:vortex-compact 165408058 / 252898478 / -34.6% 🟢 182041853 / 330851923 / -45.0% 🟢 0.91 / 0.76 / +18.9%
appian_q04/duckdb:vortex-compact 501188407 / 1578381136 / -68.2% 🟢 519110866 / 1584337025 / -67.2% 🟢 0.97 / 1.00 / -3.1%
appian_q05/duckdb:vortex-compact 174928250 / 238764360 / -26.7% 🟢 193867548 / 282660092 / -31.4% 🟢 0.90 / 0.84 / +6.8%
appian_q06/duckdb:vortex-compact 260195155 / 989183772 / -73.7% 🟢 238227815 / 1168905881 / -79.6% 🟢 1.09 / 0.85 / +29.1%
appian_q07/duckdb:vortex-compact 213983347 / 287853340 / -25.7% 🟢 232090714 / 351884874 / -34.0% 🟢 0.92 / 0.82 / +12.7%
appian_q08/duckdb:vortex-compact 710715797 / 919757488 / -22.7% 🟢 726664834 / 959031756 / -24.2% 🟢 0.98 / 0.96 / +2.0%
duckdb / parquet / ns (0.644x ✅, 7↑ 0↓)
name hot ns (PR / base / %diff) cold ns (PR / base / %diff) hot/cold (PR / base / %diff)
appian_q01/duckdb:parquet 133240362 / 156564676 / -14.9% 🟢 157401728 / 188914719 / -16.7% 🟢 0.85 / 0.83 / +2.1%
appian_q02/duckdb:parquet 329219152 / 358296061 / -8.1% 343915280 / 374541949 / -8.2% 0.96 / 0.96 / +0.1%
appian_q03/duckdb:parquet 179036901 / 225423409 / -20.6% 🟢 201604894 / 247626961 / -18.6% 🟢 0.89 / 0.91 / -2.4%
appian_q04/duckdb:parquet 503915923 / 1479249768 / -65.9% 🟢 516979697 / 1518486287 / -66.0% 🟢 0.97 / 0.97 / +0.1%
appian_q05/duckdb:parquet 203714195 / 247446991 / -17.7% 🟢 232242439 / 285929395 / -18.8% 🟢 0.88 / 0.87 / +1.4%
appian_q06/duckdb:parquet 235766971 / 985678207 / -76.1% 🟢 252768376 / 1019944856 / -75.2% 🟢 0.93 / 0.97 / -3.5%
appian_q07/duckdb:parquet 221250363 / 264777407 / -16.4% 🟢 256669862 / 281643357 / -8.9% 0.86 / 0.94 / -8.3%
appian_q08/duckdb:parquet 664356067 / 782063438 / -15.1% 🟢 688856605 / 821261102 / -16.1% 🟢 0.96 / 0.95 / +1.3%

No file size changes detected.

@github-actions

github-actions Bot commented Sep 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmarks: Random Access 📖

Commits: PR dd8b5948 vs base 792cef74
Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: +70.9%
Engines: random-access No clear signal (+70.9%, environment too noisy confidence)
Vortex (geomean): hot 0.892x ✅
Parquet (geomean): hot 0.522x ✅
Shifts: Parquet (control) -47.8% · Median polish -32.6%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.
  • Hot vs cold: Every measurement is run several times. The first run is reported as the cold run, and the median of the runs after it is reported as the hot run. The verdict and significance use hot runs; each target's geomean reports hot and cold beside each other where the individual runs were recorded and hot alone where they were not, the cold column shows first-run cost per row, and hot/cold is how much of each run the warm path saves. Rows whose results predate per-run reporting show only one value, taken from the value the runner reported.
  • Table cells: Each cell reads PR / base / %diff. The hot and cold columns mark a change as 🔴 slower or 🟢 faster once it clears this suite's threshold; anything smaller is noise here and is left unmarked, as is hot/cold, because a shift in the warm-up ratio is not a win or a loss by itself.
vortex / arrow-ipc / ns (0.977x ➖, 4↑ 9↓)
name ns (PR / base / %diff)
random-access/arrow-tokio-local-disk 92415 / 71276 / +29.7% 🔴
random-access/arrow-tokio-local-disk-footer 221873 / 204277 / +8.6%
random-access/feature-vectors/correlated/arrow-tokio-local-disk 1564619 / 3033097 / -48.4% 🟢
random-access/feature-vectors/correlated/arrow-tokio-local-disk-footer 2121423 / 3058949 / -30.6% 🟢
random-access/feature-vectors/uniform/arrow-tokio-local-disk 35011351 / 68555695 / -48.9% 🟢
random-access/feature-vectors/uniform/arrow-tokio-local-disk-footer 35283070 / 68001933 / -48.1% 🟢
random-access/nested-lists/correlated/arrow-tokio-local-disk 76766 / 67185 / +14.3% 🔴
random-access/nested-lists/correlated/arrow-tokio-local-disk-footer 140999 / 121230 / +16.3% 🔴
random-access/nested-lists/uniform/arrow-tokio-local-disk 1341719 / 1293333 / +3.7%
random-access/nested-lists/uniform/arrow-tokio-local-disk-footer 1383423 / 1380083 / +0.2%
random-access/nested-structs/correlated/arrow-tokio-local-disk 63771 / 54788 / +16.4% 🔴
random-access/nested-structs/correlated/arrow-tokio-local-disk-footer 126132 / 102652 / +22.9% 🔴
random-access/nested-structs/uniform/arrow-tokio-local-disk 1057994 / 997279 / +6.1%
random-access/nested-structs/uniform/arrow-tokio-local-disk-footer 1136730 / 1026285 / +10.8% 🔴
random-access/taxi/correlated/arrow-tokio-local-disk 157860 / 124553 / +26.7% 🔴
random-access/taxi/correlated/arrow-tokio-local-disk-footer 294585 / 270817 / +8.8%
random-access/taxi/uniform/arrow-tokio-local-disk 3496813 / 2830300 / +23.5% 🔴
random-access/taxi/uniform/arrow-tokio-local-disk-footer 3623990 / 2930271 / +23.7% 🔴
random-access / vortex-file-compressed / ns (0.892x ✅, 10↑ 1↓)
name ns (PR / base / %diff)
random-access/feature-vectors/correlated/vortex-tokio-local-disk 306082 / 324858 / -5.8%
random-access/feature-vectors/correlated/vortex-tokio-local-disk-footer 1491185 / 1658713 / -10.1% 🟢
random-access/feature-vectors/uniform/vortex-tokio-local-disk 1864294 / 2583025 / -27.8% 🟢
random-access/feature-vectors/uniform/vortex-tokio-local-disk-footer 3430247 / 4423800 / -22.5% 🟢
random-access/nested-lists/correlated/vortex-tokio-local-disk 257189 / 299144 / -14.0% 🟢
random-access/nested-lists/correlated/vortex-tokio-local-disk-footer 447898 / 461012 / -2.8%
random-access/nested-lists/uniform/vortex-tokio-local-disk 1164212 / 1644340 / -29.2% 🟢
random-access/nested-lists/uniform/vortex-tokio-local-disk-footer 1385965 / 1918670 / -27.8% 🟢
random-access/nested-structs/correlated/vortex-tokio-local-disk 355467 / 337263 / +5.4%
random-access/nested-structs/correlated/vortex-tokio-local-disk-footer 660284 / 570588 / +15.7% 🔴
random-access/nested-structs/uniform/vortex-tokio-local-disk 812080 / 977668 / -16.9% 🟢
random-access/nested-structs/uniform/vortex-tokio-local-disk-footer 1200979 / 1383977 / -13.2% 🟢
random-access/taxi/correlated/vortex-tokio-local-disk 871195 / 922488 / -5.6%
random-access/taxi/correlated/vortex-tokio-local-disk-footer 1634297 / 1727996 / -5.4%
random-access/taxi/uniform/vortex-tokio-local-disk 3211543 / 3870509 / -17.0% 🟢
random-access/taxi/uniform/vortex-tokio-local-disk-footer 4604186 / 5293963 / -13.0% 🟢
random-access/vortex-tokio-local-disk 632150 / 605317 / +4.4%
random-access/vortex-tokio-local-disk-footer 1333389 / 1257614 / +6.0%
random-access / parquet / ns (0.522x ✅, 18↑ 0↓)
name ns (PR / base / %diff)
random-access/feature-vectors/correlated/parquet-tokio-local-disk 109105169614 / 359714381816 / -69.7% 🟢
random-access/feature-vectors/correlated/parquet-tokio-local-disk-footer 110022507979 / 357249728423 / -69.2% 🟢
random-access/feature-vectors/uniform/parquet-tokio-local-disk 111290060523 / 351990005586 / -68.4% 🟢
random-access/feature-vectors/uniform/parquet-tokio-local-disk-footer 109637979498 / 352116756500 / -68.9% 🟢
random-access/nested-lists/correlated/parquet-tokio-local-disk 150644307 / 242121980 / -37.8% 🟢
random-access/nested-lists/correlated/parquet-tokio-local-disk-footer 150931751 / 243067361 / -37.9% 🟢
random-access/nested-lists/uniform/parquet-tokio-local-disk 150666505 / 238151078 / -36.7% 🟢
random-access/nested-lists/uniform/parquet-tokio-local-disk-footer 150749507 / 247501424 / -39.1% 🟢
random-access/nested-structs/correlated/parquet-tokio-local-disk 7624924 / 19535864 / -61.0% 🟢
random-access/nested-structs/correlated/parquet-tokio-local-disk-footer 7733611 / 19373960 / -60.1% 🟢
random-access/nested-structs/uniform/parquet-tokio-local-disk 7591333 / 19017682 / -60.1% 🟢
random-access/nested-structs/uniform/parquet-tokio-local-disk-footer 7784586 / 19081862 / -59.2% 🟢
random-access/parquet-tokio-local-disk 139098672 / 177787099 / -21.8% 🟢
random-access/parquet-tokio-local-disk-footer 139665316 / 178571881 / -21.8% 🟢
random-access/taxi/correlated/parquet-tokio-local-disk 208633831 / 265363702 / -21.4% 🟢
random-access/taxi/correlated/parquet-tokio-local-disk-footer 209574183 / 266125257 / -21.2% 🟢
random-access/taxi/uniform/parquet-tokio-local-disk 219825637 / 277597393 / -20.8% 🟢
random-access/taxi/uniform/parquet-tokio-local-disk-footer 220139381 / 280104705 / -21.4% 🟢
random-access / lance / ns (1.052x ➖, 0↑ 5↓)
name ns (PR / base / %diff)
random-access/feature-vectors/correlated/lance-tokio-local-disk 360326 / 359770 / +0.2%
random-access/feature-vectors/correlated/lance-tokio-local-disk-footer 1576443 / 1512182 / +4.2%
random-access/feature-vectors/uniform/lance-tokio-local-disk 1159971 / 1098615 / +5.6%
random-access/feature-vectors/uniform/lance-tokio-local-disk-footer 2418051 / 2182059 / +10.8% 🔴
random-access/lance-tokio-local-disk 646664 / 614814 / +5.2%
random-access/lance-tokio-local-disk-footer 1800611 / 1680791 / +7.1%
random-access/nested-lists/correlated/lance-tokio-local-disk 216894 / 190769 / +13.7% 🔴
random-access/nested-lists/correlated/lance-tokio-local-disk-footer 900156 / 824815 / +9.1%
random-access/nested-lists/uniform/lance-tokio-local-disk 954048 / 923059 / +3.4%
random-access/nested-lists/uniform/lance-tokio-local-disk-footer 1669993 / 1584725 / +5.4%
random-access/nested-structs/correlated/lance-tokio-local-disk 347795 / 309033 / +12.5% 🔴
random-access/nested-structs/correlated/lance-tokio-local-disk-footer 809675 / 700837 / +15.5% 🔴
random-access/nested-structs/uniform/lance-tokio-local-disk 2554653 / 2461820 / +3.8%
random-access/nested-structs/uniform/lance-tokio-local-disk-footer 2989110 / 2835181 / +5.4%
random-access/taxi/correlated/lance-tokio-local-disk 947608 / 951602 / -0.4%
random-access/taxi/correlated/lance-tokio-local-disk-footer 2410323 / 2185155 / +10.3% 🔴
random-access/taxi/uniform/lance-tokio-local-disk 9255660 / 10082745 / -8.2%
random-access/taxi/uniform/lance-tokio-local-disk-footer 10296986 / 10956455 / -6.0%

@claude
Add Graviton3 SVE miscompile repros to string-bench
7e613e8 
Two examples reproduce the Vortex file read failures seen once the
benchmarks moved to c7gd.metal (OnPair/FSST decoded bytes disagree with
uncompressed_lengths):

* `sve_widening_sum`: dependency-free. With `-C target-cpu=neoverse-v1`
  (what `target-cpu=native` resolves to on Graviton3), rustc 1.98 /
  LLVM 22 miscompiles a widening `u8 -> usize` sum at an SVE vector
  length of 256 bits or more; the same binary is correct at 128 bits,
  and u16/u32/i32 sums are correct everywhere. Both codecs size their
  decode buffer with exactly that sum over the file's `u8` string
  lengths, so the buffer comes out about half the needed size.
* `arm64_repro`: the same fault through the real path, on the ClickBench
  URL column, as five independent stages (in-memory OnPair and FSST,
  then file write/read with OnPair, FSST and the default compressor),
  with a `--dump` mode that writes every decoded child and its widening
  sum so two runs can be diffed.

Signed-off-by: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FUSFWCFtHuDq5uCqDgGm9i
@claude
Disable SVE codegen for Graviton3 benchmark builds
f59b1ec 
`-C target-cpu=native` on c7gd.metal resolves to neoverse-v1, and
rustc 1.98 / LLVM 22 then miscompiles widening `u8 -> usize` sums at
256-bit SVE. OnPair and FSST size their decode buffers with that sum
over a file's `u8` string lengths, so every SQL, compression and string
benchmark that read a Vortex file failed on the new runners. Add
`-C target-feature=-sve,-sve2` to the three benchmark build steps until
the compiler is fixed; the standalone repro is
`benchmarks/string-bench/examples/sve_widening_sum.rs`.

Signed-off-by: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FUSFWCFtHuDq5uCqDgGm9i
@claude
Gate the arm64_repro example on unstable_encodings
dd8b594 
The string-bench library is compiled out without the feature, so the
example must declare it as required, like the binary already does;
workspace builds without the feature (the musl test job) otherwise fail
to compile it.

Signed-off-by: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FUSFWCFtHuDq5uCqDgGm9i
@robert3005 robert3005 added the action/bench-all Runs all CPU benchmarks. Do not combine with other benchmark labels; GPU is the only exception. label Sep 3, 2026
@github-actions github-actions Bot removed the action/bench-all Runs all CPU benchmarks. Do not combine with other benchmark labels; GPU is the only exception. label Sep 3, 2026
@robert3005

robert3005 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

going to try #9761

@robert3005 robert3005 closed this Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

changelog/ci

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@joseph-isaacs @robert3005 @claude